Least Squares and Fourier Series

We want to find the minimum of $\int_{-\pi}^\pi|f(x)-T_N(x)|^2dx$, where $T_N\in \text{span}\{e^{inx}\}_{n=-N}^N\ $ — i.e., $T_N(x)=\sum_{n=-N}^N \beta_n e^{inx}$. In class, we used the technique from section 0.5 in the text. Here, we will use a more direct method. First of all, \[ \int_{-\pi}^\pi|f(x)-T_N(x)|^2dx = \int_{-\pi}^\pi|f(x)|^2dx -\int_{-\pi}^\pi f(x)\overline{T_N(x)}dx -\int_{-\pi}^\pi \overline{f(x)}T_N(x)dx + \int_{-N}^N |T_N(x)|^2dx \] Let's compute $\int_{-\pi}^\pi f(x)\overline{T_N(x)}dx$. Put the expression for $T_N$ into the integral and use a little calculus to get \[ \int_{-\pi}^\pi f(x)\overline{T_N(x)}dx = \sum_{n=-N}^N \int_{-\pi}^\pi f(x)\overline{\beta_n e^{inx}}dx=\sum_{n=-N}^N \overline{\beta_n} \underbrace{\int_{-\pi}^\pi f(x)e^{-inx}}_{2\pi \alpha_n} = 2\pi \sum_{n=-N}^N \overline{\beta_n}\alpha_n\] Because $\overline{\int_{-\pi}^\pi \overline{f(x)}T_N(x)dx}=\int_{-\pi}^\pi f(x)\overline{T_N(x)}dx$, we have that \[ \int_{-\pi}^\pi \overline{f(x)}T_N(x)dx = 2\pi\sum_{n=-N}^N \overline{\overline{\beta_n}\alpha_n} =2\pi \sum_{n=-N}^N\overline{\alpha_n}\beta_n \] Using a similar calculation (Can you do it?), one can also show that $\int_{-N}^N |T_N(x)|^2dx =2\pi \sum_{n=-N}^N |\beta_n|^2$. From this and the two previous formulas we see that \[ \int_{-\pi}^\pi|f(x)-T_N(x)|^2dx = \int_{-\pi}^\pi|f(x)|^2dx -2\pi\sum_{n=-N}^N \left(\overline{\beta_n}\alpha_n + \overline{\alpha_n}\beta_n\right) + 2\pi\sum_{n=-N}^N |\beta_n|^2 = \int_{-\pi}^\pi|f(x)|^2dx+2\pi\sum_{n=-N}^N\left( |\beta_n|^2-\overline{\beta_n}\alpha_n - \overline{\alpha_n}\beta_n\right) \] By completing the square, we have $|\beta_n|^2-\overline{\beta_n}\alpha_n - \overline{\alpha_n}\beta_n= |\beta_n -\alpha_n|^2-|\alpha_n|^2$. It follows that \[ \int_{-\pi}^\pi|f(x)-T_N(x)|^2dx = \int_{-\pi}^\pi|f(x)|^2dx +2\pi\sum_{n=-N}^N |\beta_n-\alpha_n|^2 - 2\pi\sum_{n=-N}^N |\alpha_n|^2. \] The middle term will be strictly positive if $\beta_n\ne\alpha_n$ even for one $n$; consequently, if $T_N(x)\ne S_N(x)$ \[ \int_{-\pi}^\pi|f(x)-T_N(x)|^2dx > \int_{-\pi}^\pi|f(x)|^2dx - 2\pi\sum_{n=-N}^N |\alpha_n|^2 = \int_{-\pi}^\pi|f(x)-S_N(x)|^2dx \] Thus the minimum occurs exactly when $T_N(x)=S_N(x)$. Moreover, the minimiizer is unique; that is,only the partial sum is the minimizer. Finally, we write down the minimum itself: \[ \int_{-\pi}^\pi|f(x)-S_N(x)|^2dx = \int_{-\pi}^\pi|f(x)|^2dx - 2\pi\sum_{n=-N}^N |\alpha_n|^2. \]