I am not sure how to do this problem. I tried to look up similar problems but was unsuccessful at finding any. I would appreciate if someone could show me how to solve this problem. Here's the question again :
Find constants a and b for which $F(a,b) =\int_0^\pi \{ \sin x - (ax^2 + bx)\}^2 dx$ is a minimum. (You may use Wolfram or other tools to integrate)
The function is $$F(a,b) = {{10\,\pi^3\,b^2+\left(15\,\pi^4\,a-60\,\pi\right)\,b+6\,\pi^5\,a^2 +\left(240-60\,\pi^2\right)\,a+15\,\pi}\over{30}}.$$ And the critical points are given by: $$\frac{\partial F(a,b)}{\partial a} = {{15\,\pi^4\,b+12\,\pi^5\,a-60\,\pi^2+240}\over{30}} = 0,$$
$$\frac{\partial F(a,b)}{\partial b} = {{20\,\pi^3\,b+15\,\pi^4\,a-60\,\pi}\over{30}} = 0,$$ that is a $2\times 2$ linear system.