How to get bias-variance expression on linear regression with p parameters?

36 Views Asked by At

How to prove that

$Err(x_0) = \textbf{E}(Y - \hat{f}_p(x_0))^2\mid X = x_0) = \sigma^2_\epsilon + (f(x_0)-\textbf{E}\hat{f}_p(x_0))^2 + \mid\mid \textbf{h}(x_0)\mid\mid^2 \sigma^2 _\epsilon$,

where

$\textbf{h}(x_0) = X(X^\top X)^{-1}x_0$, $\hat{f}_p(x) = x^\top \beta$?

I started writing something like this:

$Err(x_0) = \textbf{E}(Y - \textbf{E}\hat{f}_p(x_0))^2 = \textbf{E}(Y - \textbf{E}(x_0^\top \hat{\beta}))^2 = \textbf{E}(Y^2 + \textbf{E}(x_0^\top \hat{\beta})^2 + 2Yx_0^\top \hat{\beta})$

where

$\textbf{E}(Y^2) =var(Y) + \textbf{E}(Y)^2 = \sigma^2_\epsilon + \hat{f}_p(x_0)$

$\textbf{E}(\textbf{E}(x_0^\top \hat{\beta})^2) = var(\textbf{E}(x_0^\top \hat{\beta})) + \textbf{E}(\textbf{E}(x_0^\top \hat{\beta}))^2$

What I should do next?