Minimize quadratic deviation

Question

We need to find $p,q$ such that $$ \min_{a,b\in\mathbb{R}}\mathbb{E}[|Y-aX-b|^2]=\mathbb{E}[|Y-pX-q|^2] $$ A bit of calculus gives you the first derivates if we look at $\mathbb{E}[|Y-aX-b|^2]$ as a function $f(a,b)=\mathbb{E}[|Y-aX-b|^2]$ $$ f_a=-2\mathbb{E}(X,Y)+2a\mathbb{E}(X^2)+2b\mathbb{E}(X) $$ and $$ f_b=-2\mathbb{E}(Y)+2a\mathbb{E}(X)+2b $$ Now I put $(f_a,f_b)^T=(0,0)^T$ and I solved this and got $$ a=\frac{Cov(X,Y)}{Var(x)}; b=\mathbb{E}(Y)-\frac{Cov(X,Y)}{Var(x)}\mathbb{E}(X) $$ One can write $a,b$ also in terms of standard-deviation and the correlation coefficient. Ok. Now I want to show that $a,b$ is really a minimum. Normally I would calculate the Hessian-Matrix. $$ f_{aa}=2\mathbb{E}(X^2); f_{ab}=2\mathbb{E}(X); f_{bb}=2; f_{ba}=2\mathbb{E}(X) $$ Now my questions are: 1.) Nearly everything is dependent on $\mathbb{E}(X^2)\neq0$ right? How do I show that? 2.) How do I show that $a,b$ is really minimize $\mathbb{E}[|Y-aX-b|^2]$?

Answer 1

If $E(X^2)=0$, then $X^2=0$ almost everywhere, that is it is almost $0$ everywhere. In that case, just let $b = E(Y)$.

We have to show that the Hessian is positive semidefinite.

That is we want to show that $$E(X^2)-E(X)^2 \ge 0$$

but that is equivalent to $$Var(X) \ge 0.$$