- Suppose we want to make the approximation $$ x \approx \frac{1}{2} a_{0}+a_{1} \cos x+a_{2} \cos (2 x)+a_{3} \cos (3 x), \quad 0 \leq x \leq \pi $$ in such a way that the $L^{2}$ norm of the error on $0 \leq x \leq \pi$ is minimized.
$ (a)$ What values of $a_{0}, a_{1}, a_{2}, a_{3}$ accomplish this?
$ (b)$ What is this minimum $L^{2}$ error? (Note: $L^{2}$ error calculations normally require numerical integration$^{8}$, because symbolic integration is very difficult and therefore slow.)
I am a physics major and I still don't know how to solve the $L^2$ approximation problem. It would be very helpful if someone could show me how to do it.
Suppose that $M$ is a subspace of an inner product space $X$, and let $x\in X$ be given. In order to find the closest $m$ to $x$, it is necessary and sufficient to find $m \in M$ such that $(x-m)\perp M$, meanining that $\langle x-m,m'\rangle=0$ for all $m'\in M$. In other words, the closest point projection is the same as the orthogonal projection. To see why this is true, suppose that $(x-m)\perp M$. Then, for every $m'\in M$, $$ (x-m')=(x-m)+(m-m') $$ and $(x-m)\perp (m-m')$ because $(x-m)\perp M$. Therefore, $$ \|x-m'\|^2=\|x-m\|^2+\|m-m'\|^2 \ge \|x-m\|^2 $$ and equality holds iff $\|m-m'\|=0$ or, equivalently, $m=m'$. So an orthogonal projection is always closest, and an orthogonal projection is unique if it exists.
In your case, $M=[\{1,\cos(x),\cos(2x),\cos(3x)\}]$ is the subspace spanned by the four given functions. The closest point projection of the function $x$ onto $M$ is the unique $\alpha + \beta\cos(x)+\gamma\cos(2x)+\delta\cos(3x)$ such that $$ x-\alpha-\beta\cos(x)-\gamma\cos(2x)-\delta\cos(3x) $$ is orthogonal to $1,\cos(x),\cos(2x),\cos(3x)$ in $L^2[0,\pi]$, which gives you a system of $4$ equations in the $4$ unknowns $\alpha,\beta,\gamma,\delta$. This system is simplified by the mutual orthogonality of $1,\cos(x),\cos(2x),\cos(3x)$ in $L^2[0,\pi]$.