Solutions to Least Squares Problems

P0(x)=1
P1(x)=x
P2(x)=3x2 - 1

Solution. This requires doing three integrals on the interval [-1,1], and seeing that they are 0.

Problem 2. From first principles (i.e., minimize the error directly; don't just quote the answer), use these polynomials to find the best least-squares fit to exp(-x) on the interval [-1, 1].

Solution. We need to find the coefficients a₀, a₁, and a₂ that minimize the "sum" of squares

E(a₀, a₁, a₂) = _-1S ¹ (e^-x - a₀P₀(x) - a₁ P₁(x) - a₂P₂)² dx.

Squaring out the integrand, we obtain

E(a₀, a₁, a₂) = < e^-x , e^-x > + a₀²< P₀ , P₀ > + a₁²< P₁ , P₁ > + a₂²< P₂ , P₂ >
- 2a₀< e^-x , P₀ > - 2a₁< e^-x , P₁ > - 2a₂< e^-x , P₂ >
+ 2a₀a₁< P₀, P₁ > + 2a₀a₂< P₀, P₂ > + 2a₁a₂< P₁, P₂ >

The last three terms on the right above are zero, by orthogonality. The remaining integrals are straightforward calculations. Carring them all out results in this expression for the error:

E(a₀, a₁, a₂) = (1/2)(e² - e^-2) + 2a₀² + (2/3)a₁² + (8/5)a₂²
- 2(e - e^-1)a₀ - 4e^-1a₁ - 4(e- 7e^-1)a₂

Minimizing E determines the values of the a's. These values for the a's then give us the least squares fit,

L(x)=(1/2)(e - e^-1) P₀(x) - 3e^-1 P₁(x) + (5/4)(e- 7e^-1)P₂(x).

Using powers of x, this becomes

L(X)= (3/4)(11e^-1 - e) - 3e^-1x + (15/4)(e-7e^-1)x².

Problem 3. On the same set of axes, sketch exp(-x), the least-squares fit you've found, and the quadratic Taylor polynomial about x=0 for exp(-x).

Solution. The matlab code below generates the plot that follows.

x=-1:0.01:1;
L=(15/4)*(exp(1)-7*exp(-1))*x.^2-3*exp(-1)*x + (33*exp(-1)-3*exp(1))/4;
T=1-x+(1/2)*x.^2;
E=exp(-x);
plot(x,L,':',x,T,'-.',x,E), grid
title('Comparison of Least-Squares Fit and Taylor Polynomial')
legend('Least-squares', 'Taylor','e^{-x}')
xlabel('x')
ylabel('y')