Statistics (Regression Analysis): Show that the residuals from a linear regression model can be expressed as $e=(I-H) \epsilon$

Question

Statistics (Regression Analysis): Show that the residuals from a linear regression model can be expressed as $\mathbf{e} = (\mathbf{I}-\mathbf{H})\mathbf{\epsilon}$

The bold represents vectors or matrices.

I know that $\mathbf{e} = \mathbf{y}-\mathbf{Hy}$

So I tried expanding this to,

$$\mathbf{e} = \mathbf{X}\mathbf{\beta} + \epsilon - \mathbf{H X \beta - H\epsilon}$$

At this point I can see how to derive the more traditional,

$$\mathbf{e = (I-H)y}$$

I just don't understand how to solve the original problem. Thank you in advance.

Answer 1

Recall that $H$ is an orthogonal projection onto the space spanned by the columns of $X$, hence $HX\beta=X\beta$, thus \begin{align} e & = Y - \hat{Y} \\ &= X\beta + \epsilon - HY \\ & = X\beta + \epsilon - H(X\beta + \epsilon) \\ & = X\beta - X\beta + \epsilon - H \epsilon \\ & = (I-H)\epsilon. \end{align}