Find $a,b,c$ that minimize $F(a,b,c) = \int_{-1}^1 (t^2-a-bt-c\cos t)^2 dt$
I'm studying numerical methods and we're seeing least squares method and projection of things like polynomials and orthogonality and so. How should I minimize that thing in this context?
I think the first thing I should do it to see this as the orthogonal projection of something onto something like here. Can somebody help me?
This is a special case of the problem of minimising $F(a):=\int_S (\sum_i a_i f_i(t)-f(t))^2dt$ viz. $0=\frac{1}{2}\frac{\partial F}{\partial a_i}=\int_S(\sum_j a_j f_j-f)f_idt$. Writing $M_{ij}:=\int_S f_if_j dt,\,v_i:=\int_S ff_i dt$, we have the matrix equation $Ma=v$ i.e. $a=M^{-1}v$. In the case at hand we get $$\left(\begin{array}{c} a\\ b\\ c \end{array}\right)=\left(\begin{array}{ccc} \int_{-1}^{1}dt & \int_{-1}^{1}tdt & \int_{-1}^{1}\cos tdt\\ \int_{-1}^{1}tdt & \int_{-1}^{1}t^{2}dt & \int_{-1}^{1}t\cos tdt\\ \int_{-1}^{1}\cos tdt & \int_{-1}^{1}t\cos tdt & \int_{-1}^{1}\cos^{2}tdt \end{array}\right)^{-1}\left(\begin{array}{c} \int_{-1}^{1}t^{2}dt\\ \int_{-1}^{1}t^{3}dt\\ \int_{-1}^{1}t^{2}\cos tdt \end{array}\right).$$I'll leave you to compute the integrals (which vanish if they have odd integrands), invert the matrix etc.
A geometric projection gives the same result. The general idea is to write $t^2=a+bt+c\cos t+g$ for some function "orthogonal" to each of $1,\,t,\,\cos t$ in the sense $$\int_{-1}^1 gdt=\int_{-1}^1 tgdt=\int_{-1}^1 g(t)\cos t dt=0.$$But even if you can identify such a $g$, the fact that $1,\,t,\,\cos t$ aren't orthogonal complicates further progress from taking inner products viz.$$\int_{-1}^1 t^2dt=a\int_{-1}^1 dt+b\int_{-1}^1 tdt+c\int_{-1}^1 \cos tdt$$etc. Oh, it works; it's just you get the above matrix inversion problem, complicated by its non-zero off-diagonal term $\int_{-1}^1\cos tdt$. Luckily, we know how to invert $3\times 3$ matrices.