fundamental identity proof linear models

132 Views Asked by At

Prove the equation $||y - \bar{y}\boldsymbol{1}||^2 = ||\hat{y} - \bar{y}\boldsymbol{1}||^2 + ||y - \hat{y}||^2$.

\begin{align} y &= \begin{bmatrix} y_{1} \\ y_{2} \\ \vdots \\ y_{m} \end{bmatrix} \end{align}

\begin{align} 1 &= \begin{bmatrix} 1 \\ 1 \\ \vdots \\ 1 \end{bmatrix} \end{align} \begin{align} \bar{y}\boldsymbol{1} &= \begin{bmatrix} \bar{y} \\ \bar{y} \\ \vdots \\ \bar{y} \end{bmatrix} \end{align} \begin{align} \hat{y} &= \begin{bmatrix} \hat{y_{1}} \\ \hat{y_{2}} \\ \vdots \\ \hat{y_{3}} \end{bmatrix} \end{align} I tried proving $(\hat{y} - \bar{y}\boldsymbol{1}).(y - \hat{y})=0$ but not sure how to simplify.

1

There are 1 best solutions below

0
On BEST ANSWER

Indeed $(\hat{y} - \bar{y} \mathbf{1})^\top (y - \hat{y}) = 0$. To show this you need to use the properties of $\hat{y}$.


Abstractly, $\hat{y}$ is the orthogonal projection of $y$ onto the column space of $X$ (the design matrix), and $X$ is assumed to contain $\mathbf{1}$ as a column. An important property of the orthogonal projection is that $\hat{y} \in \text{colspace}(X)$ and $y - \hat{y}$ is orthogonal to everything in $\text{colspace}(X)$. This first fact implies $\hat{y} - \bar{y} \mathbf{1} \in C(X)$ and the second fact thus implies the desired statement $(\hat{y} - \bar{y} \mathbf{1})^\top (y - \hat{y}) = 0$.


More concretely, you may write $\hat{y} = X \hat{\beta}$ where $\hat{\beta}$ satisfies $$X^\top X \hat{\beta} = X^\top y. \tag{$*$}$$ Thus $X^\top \hat{y} = X^\top y$, and now use the fact that $X$ contains $\mathbf{1}$ as a column to conclude $$\mathbf{1}^\top \hat{y} = \mathbf{1}^\top y.$$ Similarly, you can use ($*$) to show $$\hat{y}^\top (y - \hat{y}) = 0.$$ Combining these last two facts yields the desired equality $(\hat{y} - \bar{y} \mathbf{1})^\top (y - \hat{y}) = 0$.