An example of minimizing Mean Square Error (MSE)

391 Views Asked by At

I've established that the function $f(x)$ that minimizes $$ MSE = \mathbb{E} \left[ (Y - f(x))^2 \right] $$

is $$f(x) = \mathbb{E} \left[ Y \mid X \right]$$

The next part of the question (from Shumway and Stoffer, 3.14) is to "apply the above result to the model:"

$$ Y = X^2 + W $$

where $X$ and $W$ are independent, and both $N(0,1)$. Additionally, I'm to "show that $MSE = 1$".

It's been a while since I've done anything related to "mathematical statistics" so I'm like a fish out of water right now. I'm not even sure where to begin. I can tell right away that the sum of a chi-squared and normal is not going to be pretty. Is there some trick with $\mathbb{E} \left[ Y \mid X \right]$ here that I'm not seeing? I'm not sure that integral calculation will be a pleasant experience...

Many thanks for any solution/first steps/pointers.

1

There are 1 best solutions below

0
On BEST ANSWER

MSE is also known as risk, and is the expected value of the loss function.

$$\begin{split}MSE&=E[(Y-f(x))^2]\\ &=E[(X^2+W-E(X^2+W|X))^2] \text{ plug in Y in both places}\\ &=E([X^2+W-E(X^2|X)-E(W|X)]^2)\text { linearity of expectation}\\ &=E[(X^2+W-X^2-E(W))^2]\text { $E(g(X)|X)=g(X)$ property, and W is independent of X}\\ &=E(W-E(W))^2 \text { I believe you can see}\\ &=var(W) \text{ definition of variance}\\ &=1\end{split}$$