R-squared and variance relation

79 Views Asked by At

According to Wiki:

https://en.wikipedia.org/wiki/Fraction_of_variance_unexplained

$1 - R^2 = VAR_{err}/VAR_{tot}$

Where $VAR_{err} = \sum_{i = 1}^N (y_i - \hat{y}_i)^2$ is the variance of the residuals.

I don't see how this is correct, unless the residuals have mean zero. To see this, define the residuals:

$e_i = y_i - \hat{y}_i$

The variance of the residuals is:

$\sum_{i = 1}^N (e_i - \bar{e}_i)^2 = \sum_{i = 1}^N (y_i - \hat{y}_i - \bar{e}_i)^2 $

Do you disagree or is Wikipedia wrong?

1

There are 1 best solutions below

2
On

Note that in OLS regression with an intercept term, due to the "first order condition", when you take derivative w.r.t. $\beta_0$ you have $$ -2\sum_{i=1}^n (y_i - \hat{\beta}_0 - \hat{\beta}_1x_{1i} - \cdots \hat{\beta}_px_{pi}) = -2\sum e_i =0, $$ hence $$ \bar{e}_n = 0, $$ thus $$ \hat{\sigma}^2 = \frac{1}{n}\sum(e_i - \bar{e})^2 = \frac{\sum e_i ^ 2}{n}. $$