Minimum Residual Energy
In the notes
on
signal spaces we commented on interpreting the integral
$\int_{-\pi}^\pi|f(t)|^2dt$ as the energy in a signal. The question
that we want to answer here is what "trig polynomial"
$T_N(t)=A_0+\sum_{k=1}^N A_k \cos(kt)+B_k\sin(kt)$ will give the best
approximation to $f$, in the sense of minimizing the residual energy,
\[
(\dagger) \quad \int_{-\pi}^\pi(f(t)-T_N(t))^2dt.
\]
We will begin by showing that functions for that
\[
(\ast) \quad \int_{-\pi}^\pi f(t)T_N(t)dt = 2\pi a_0A_0 +
\pi \sum_{k=1}^N a_k A_k+b_k B_k.
\]
To see this, note that
\[
\begin{align}
\int_{-\pi}^\pi f(t)T_N(t)dt =& \int_{-\pi}^\pi f(t)
\bigg(A_0+\sum_{k=1}^N A_k
\cos(kt)+B_k\sin(kt)\bigg)dt \\
=& A_0\int_{-\pi}^\pi f(t)dt +
\sum_{k=1}^N A_k \int_{-\pi}^\pi f(t)\cos(kt)dt +
B_k \int_{-\pi}^\pi f(t)\sin(kt)dt.
\end{align}
\]
Recall that $a_0=\frac{1}{2\pi}\int_{-\pi}^\pi f(t)dt$, and for $k\ge
1$, $a_k = \frac{1}{\pi}\int_{-\pi}^\pi f(t)\cos(kt)dt$. Similarly,
$b_k = \frac{1}{\pi}\int_{-\pi}^\pi f(t)\sin(kt)dt$. Using these in
the previous equation results in $(\ast)$. Since $f$ is arbitrary, we
may replace it by $T_N$. Doing so in $(\ast)$ results in the formulas
below:
\[
(\ast \ast)\quad \int_{-\pi}^\pi T_N(t)^2dt =2\pi A_0^2+ \pi
\sum_{k=1}^N A_k^2+B_k^2.
\]
We can now compute the residual energy defined in $(\dagger)$.
\[
\begin{align}
\int_{-\pi}^\pi(f(t)-T_N(t))^2dt =& \int_{-\pi}^\pi f(t)^2dt -
2\int_{-\pi}^\pi f(t)T_N(t)dt +
\int_{-\pi}^\pi T_N(t))^2dt \\
=& \int_{-\pi}^\pi f(t)^2dt -4\pi a_0A_0 -
2\pi \sum_{k=1}^N \big(a_k A_k+b_k B_k\big) + 2\pi A_0^2+ \pi
\sum_{k=1}^N\big( A_k^2+B_k^2)\\.
\end{align}
\]
Next, complete the square for the terms on the right in the last line
above: $2\pi(A_0^2 -2a_0 A_0) = 2\pi (A_0-a_0)^2 - 2\pi a_0^2$, $\pi
(A_k^2 -A_k) = \pi (A_k-a_k)^2 - \pi a_k^2$, and finally $\pi (B_k^2
-b_k) = \pi (B_k-b_k)^2 - \pi b_k^2$. The result is
\[
\int_{-\pi}^\pi(f(t)-T_N(t))^2dt = \int_{-\pi}^\pi f(t)^2dt +
2\pi (A_0^2-a_0)^2+ \pi \sum_{k=1}^N \big (A_k-a_k)^2 +
\pi (B_k-b_k)^2\big) - 2\pi a_0^2 -\pi \sum_{k=1}^N (a_k^2+b_k^2)
\]
The middle term will be strictly positive if $A_k\ne a_k$ and $B_k\ne
b_k$, even for one $k$; consequently, if $T_N(t)\ne S_N(t)$,
\[
\int_{-\pi}^\pi(f(t)-T_N(t))^2dt > \int_{-\pi}^\pi f(t)^2dt -2\pi
a_0^2 -\pi \sum_{k=1}^N (a_k^2+b_k^2)=
\int_{-\pi}^\pi (f(t)-S_N(t))^2dt
\]
Thus the minimum occurs exactly when $T_N(t)=S_N(t)$. Moreover, the
minimizer is unique; that is,only the partial sum $S_N$ is the
minimizer. Using the two terms on the right above, we see that the
minimum residual energy is given by
\[ (\ddagger)\quad \int_{-\pi}^\pi(f(t)-S_N(t))^2dt = \int_{-\pi}^\pi
f(t)^2dt -2\pi a_0^2 -\pi \sum_{k=1}^N (a_k^2+b_k^2) = \int_{-\pi}^\pi
f(t)^2dt - \int_{-\pi}^\pi S_N(t)^2dt\]
Convergence in the mean
We say that the partial sum $S_N$ converges to $f$ in the mean if and
only if $\lim_{N\to \infty}\int_{-\pi}^\pi(F(t)- S_N(t)^2dt
=0$. Using the expression for the minimal energy in $(\ddagger)$ we
have the following proposition.
Theorem (Parseval) The partial sum $S_N$ converges in the mean to $f$
if and only if Parseval's equation holds:
\[
(\S) \quad \int_{-\pi}^\pi f(t)^2dt
= 2\pi a_0^2 +\pi \sum_{k=1}^\infty (a_k^2+b_k^2)
\]
Proof By $(\ddagger)$, if $S_N$ converges in the mean to $f$, then
\[
0=\lim_{N\to \infty} \int_{-\pi}^\pi(f(t)-S_N(t))^2dt =\int_{-\pi}^\pi
f(t)^2dt -2\pi a_0^2 -\pi \lim_{N\to \infty}\sum_{k=1}^N (a_k^2+b_k^2),
\]
so Parseval's equation holds. Conversely, if $2\pi a_0^2 +\pi
\sum_{k=1}^\infty (a_k^2+b_k^2) = \int_{-\pi}^\pi f(t)^2dt$, then,
again by $(\ddagger)$, $S_N$ converges to $f$ in the mean.
Theorem (Riesz-Fischer) Suppose that $f$ is in
$L^2[-\pi,\pi]$. The $S_N$ converges in the mean to $f$. Conversely,
suppose that the sequence $\{a_0, a_1, b_1, a_2,b_2, \ldots a_k,b_k
\dots\} $ satisfies $2\pi a_0^2 +\pi \sum_{k=1}^\infty
(a_k^2+b_k^2)<\infty$, then there exists a unique $f$ in
$L^2[-\pi,\pi]$ such that $S_N(t)= a_0+\sum_{k=1}^N a_k
\cos(kt)+b_k\sin(kt)$ converges to $f$ in the mean.
The proof of this theorem is beyond the scope of this course. To
describe what it says we need to know the meaning of Parseval's
equation $(\S)$. Remember that the left side is interpreted as the
energy in the signal $f$. But what about the right side? If consider a
single term of the form $a_k \cos(kt)$, then the energy in this term
$2\pi a_0^2$, for $k=0$, and $\pi a_k^2$ for $k\ge 1$. The same is
true for the $b_k$'s. The energy in $b_k\sin(kt)$ is $\pi b_k^2$. The
point is that Parseval's equation $(\S)$ represents conservation
of energy. The energy in the signal is the sum of the energies in
the individual frequencies.
The Riesz-Fischer Theorem has an interesting interpretation. The
first is that for any finite energy signal, the energy in the sum of
its frequencies components equals the energy in the signal, as
mentioned above. But the reverse is also true. Given an sequence of
$a$'s and $b$'s such that the sum of the energies in the individual
frequencies is finite, then there is a unique signal $f$, with
finite energy, having the $a$'s and $b$'s from the sequence as its
Fourier series.
Let's think of this in another way. Each of the terms
$a_k\cos(kt)$ or $ b_k\sin(kt)$ represents a signal with pure
(angular) frequency $k$. The amplitude of each of these, $|a_k|$ or
$|b_k|$ represents the intensity of a frequency — for example,
how loud the sound corresponding to the frequency component is. What
the Riesz-Fischer Theorem implies is that for any sequence of $a$'s
and $b$'s, as long as the sum $2\pi a_0^2 +\pi \sum_{k=1}^\infty
(a_k^2+b_k^2)<\infty$, there will be a corresponding finite energy
signal. Put another way, you can synthesize a signal
with arbitrary frequencies in it, provided the total energy in the
components is finite.