Least squares approximation: Legendre polynomial

Question

Find the best quadratic least squares approximation to $f(x)=e^x$ on $[-1,1]$ with respect to the inner product $\langle f(x),g(x) \rangle = \displaystyle\int_{-1}^1 f(x)g(x)dx$.

I cannot figure out how to do this. Is there a good online resource for this? Unfortunately, I don't have a book for it.

Any tips/hints would be helpful.

Answer 1

Let $X$ be the inner product space in question. Note $\dim P_2 =3$. The problem is $\min_{p \in P_2} \|p-\exp\|^2$.

Note that the minimum exists by a compactness argument because $P_2$ is finite dimensional and $\|p-\exp\| \ge \|p\|-\|\exp\|$, hence has a solution $\hat{p} \in P_2$.

It follows that $(\hat{p}-\exp) \bot P_2$ (and hence that $\hat{p}$ is unique by Pythagoras).

In other words, you need to find the $\hat{p}$ that solves $\langle \hat{p}-\exp, p \rangle = 0$ for all $p \in P_2$.

Note that if $b_0,b_1,b_2$ is any basis of $P_2$, this reduces to solving $\langle \hat{p}-\exp, b_k \rangle = 0$, or $\langle \hat{p}, b_k \rangle = \langle \exp, b_k \rangle $.

Since $\hat{p} \in P_2$, we can write $\hat{p} = \sum_i \pi_i b_i $ for some constants $\pi_i$, so the problem reduces to solving $\sum_i \pi_i \langle b_i, b_k \rangle = \langle \exp, b_k \rangle $.

If the $b_k$ are orthogonal, the solution is immediate, otherwise this reduces to solving a $3 \times 3$ system of equations.

The Legendre polynomials $P_k$ are orthogonal, hence letting $b_k =P_k$, we obtain $\pi_k = { \langle \exp, P_k \rangle \over \|P_k\|^2 }$.

Answer 2

You have to project $e^x$ on the vector space $P_2 = \{ ax^2+bx+c : a,b,c \in \mathbb{R} \}$.

Notice that a basis for $P_2$ is $1,x, x^2$. So you have to find $a,b,c \in \mathbb{R}$ such that $\langle e^x - (ax^2+bx+c), 1 \rangle = \langle e^x- (ax^2+bx+c), x \rangle = \langle e^x- (ax^2+bx+c), x^2 \rangle = 0$.

So you have three equations with three indeterminates $a,b,c$

$$ \int_{-1}^1 (e^x - (ax^2+bx+c)) \ dx=0 $$ $$ \int_{-1}^1 (e^x - (ax^2+bx+c))x \ dx=0 $$ $$ \int_{-1}^1 (e^x - (ax^2+bx+c))x^2 \ dx=0 $$

Getting $$ \left\{ \begin{matrix} -\frac{2}{3} a &&-2c &+e - \frac{1}{e} &=& 0 \\ & -\frac{2}{3} b && +\frac{2}{e} &=& 0 \\ -\frac{2}{5} a && -\frac{2}{3} c &+e - \frac{5}{e} &=& 0 \end{matrix} \right. $$