Let $X,Y_1,Y_2$ be elements of the Hilbert space $L²$, and $Y$ is to be approximated by
$$\hat{Y}=aX_1+bX_2$$
which minimizes the mean squared error
$$MSE:=\mathbb{E}|Y-\hat{Y}|^2.$$
Now I have to find $a$ and $b$.
What have I done? I know that for $X,Y\in L²$ the mean $\mathbb{E}(XY)=\langle X,Y\rangle.$ First, I've written the MSE into
$$MSE=\mathbb{E}Y^2+a^2\mathbb{E}X_1^2+b^2\mathbb{E}X_2^2-2(a\mathbb{E}(YX_1)+b\mathbb{E}(YX_2))+2ab\mathbb{E}(X_1X_2)$$
Second, I've derivatived $MSE$ with respect to $a$ and $b$, i.e. \begin{align*} \frac{\partial MSE}{\partial a}&=2a\mathbb{E}(X_1^2)-2\mathbb{E}(YX_1)+2b\mathbb{E}(X_1X_2)=0\\ \frac{\partial MSE}{\partial b}&=2b\mathbb{E}(X_2^2)-2\mathbb{E}(YX_2)+2a\mathbb{E}(X_1X_2)=0 \end{align*} This is equal to \begin{align*} a\mathbb{E}X_1^2+b\mathbb{E}(X_1X_2) &= \mathbb{E}(YX_1)\\ a\mathbb{E}(X_1X_2)+b\mathbb{E}X_2^2 &=\mathbb{E}(YX_2). \end{align*} UPDATE:
Next, I form the above linear system into a matrix
$$ \begin{pmatrix} \mathbb{E}X_1^2 & \mathbb{E}(X_1X_2)\\ \mathbb{E}(X_1X_2) & \mathbb{E}X_2^2 \end{pmatrix} \cdot\begin{pmatrix} a\\ b \end{pmatrix} =\begin{pmatrix} \mathbb{E}(YX_1)\\ \mathbb{E}(YX_2) \end{pmatrix} $$ Then, after building the inverse, I get my solutions, i.e.
$$ \begin{pmatrix} a\\ b \end{pmatrix}= \frac{1}{\mathbb{E}X_1^2\mathbb{E}X_2^2-\mathbb{E}(X_1X_2)^2} \begin{pmatrix} \mathbb{E}X_2^2 & - \mathbb{E}(X_1X_2)\\ -\mathbb{E}(X_1X_2) & \mathbb{E}X_1^2 \end{pmatrix} \cdot \begin{pmatrix} \mathbb{E}(YX_1)\\ \mathbb{E}(YX_2) \end{pmatrix} $$
So, I get $$ \begin{align*} a &= \frac{\mathbb{E}X_2^2\mathbb{E}(YX_1)-\mathbb{E}(X_1X_2)\mathbb{E}(YX_2)}{\mathbb{E}X_1^2\mathbb{E}X_2^2-\mathbb{E}(X_1X_2)^2}\\ b &= \frac{\mathbb{E}X_1^2\mathbb{E}(YX_2)-\mathbb{E}(X_1X_2)\mathbb{E}(YX_1)}{\mathbb{E}X_1^2\mathbb{E}X_2^2-\mathbb{E}(X_1X_2)^2} \end{align*} $$