How to show $\sum_i(\bar{y}-\hat{y_i})e_i=0$

Question

How to show $\sum_i(\bar{y}-\hat{y_i})e_i=0$, I have no trouble to see $$\sum_i(\bar{y}-\hat{y_i})=0,$$ I got stuck at $e_i$ is a random variable, not a constant.

Appreciate any comment

Answer 1

Let $\mathbf{y} = (y_1, \cdots, y_n)$ and similarly defined for $\widehat{\mathbf{y}}$. By definition of regression, we have that $\widehat{\mathbf{y}} = \mathbf{H} \mathbf{y}$, where $\mathbf{H} = \mathbf{X}(\mathbf{X}^\intercal \mathbf{X})^{-1}\mathbf{X}^\intercal$. Notice that $\mathbf{H} = \mathbf{H}^\intercal$ and $\mathbf{H}^2 = \mathbf{H}$. \begin{align*} \sum_i(\overline{y} - \widehat{y}_i)e_i &= (\overline{y}\mathbf{1} - \mathbf{H}\mathbf{y})^\intercal \mathbf{e} \\ &=\left(\frac{1}{n}\mathbf{1}\mathbf{1}^\intercal \mathbf{y} - \mathbf{H}\mathbf{y}\right)^\intercal \left(\mathbf{y} - \mathbf{H}\mathbf{y}\right) \\ &=\mathbf{y}^\intercal\left(\frac{1}{n}\mathbf{1}\mathbf{1}^\intercal - \mathbf{H}\right)\left(\mathbf{I} - \mathbf{H}\right)\mathbf{y} \\ &=\mathbf{y}^\intercal\left(\frac{1}{n}\mathbf{1}\mathbf{1}^\intercal\right)\left(\mathbf{I} - \mathbf{H}\right)\mathbf{y} - \mathbf{y}\mathbf{H}\left(\mathbf{I} - \mathbf{H}\right)\mathbf{y} \\ &= \mathbf{y}^\intercal\left(\frac{1}{n}\mathbf{1}\mathbf{1}^\intercal\right)\left(\mathbf{I} - \mathbf{H}\right)\mathbf{y} - \mathbf{y}\left(\mathbf{H} - \mathbf{H}^2\right)\mathbf{y} \\ &= \mathbf{y}^\intercal\left(\frac{1}{n}\mathbf{1}\mathbf{1}^\intercal\right)\left(\mathbf{I} - \mathbf{H}\right)\mathbf{y} \\ &= \overline{y} \sum_i(y_i - \widehat{y}_i) \\ &= 0 \end{align*}

Answer 2

Basically, there are two parts in this equation, since $\sum (\bar{y} - \hat y_i )e_i = \bar y \sum e_i - \sum \hat y_i e_i = 0$, thus we need to show that $\sum e_i = 0$ and $\sum \hat y_i e_i = 0$, which follows straight from the first order condition of the OLS. Namely, $\sum e_i = 0$ stems from the derivative of $\sum (y_i - \sum_k \beta_0 - \beta_j x_j)^2$ w.r.t. $\beta_0$, and $\sum \hat y_i e_i = 0$ from the orthogonality of $\hat y$ and $e$. You can show it, WLOG, by using the $j$th derivative of $\sum (y_i - \sum_k \beta_0 - \beta_j x_j)^2$ to show that the error term $e$ is orthogonal to each one of the $x$, and therefore to $\hat y$, since $\hat y$ is a linear combination of the $x$s.