What is the most efficient way to calculate $R^2$?

Question

Hello I am working on a question from an old exam paper and wondered what is the best way to tackle parts ii and iii. Given the data it is easy to find $\hat{\beta_0}=-1.071$ and $\hat{\beta_1}=2.741$.

Now for part ii) I have the formula $R^2=1-SSE/SST$ where $SST=\sum(y_i-\bar{y})^2$ (easy to work out) and $SSE=\sum e_i^2=\sum (y_i-\hat{y_i})$.

Likewise I have for part iii) An unbiased estimate of $\sigma^2$ is $\sum e_i^2/(n-2)$.

Question: I wondered if there is a nice and more efficient way to work out $\sum e_i^2$ or do I have to calculate each predicted value based on the model take it away from the actual value square that value and then sum all the values up?

Answer 1

Recall that in simple linear regression the square of the Pearson correlation coefficient equals $R^2$, thus $$ r^2 = \left( \frac{\sum x_i y_i -n\bar{x}\bar{y}}{(\sum x_i^2 - n\bar{x}^2)^{1/2}(\sum y_i^2 - n\bar{y}^2)^{1/2}} \right)^2=\hat{\beta}_1^2\frac{S_{XY}}{S_{YY}}=R^2. $$ Which, given the table, shouldn't be hard to compute.

Answer 2

Yes, you are correct that computing SSE using the definition is a tedious job. If the regression coefficients are estimated without rounding errors, then the following procedure may be used for computing SSE.

By definition, $SSE = \sum(y_i-\hat{y_i})^2$, and $\hat{y}_i=\hat{\beta}_0 + \hat{\beta}_1 x$.

Plug-in $\hat{y_i}$ into the definition of SSE and simplify. It results in the following: $$SSE = \sum y_{i}^{2}-\hat{\beta_0}\sum y_i - \hat{\beta_1}\sum x_i y_i.$$

Answer 3

There are many perspectives on 'efficiency' of computation for regression. Often a data file and software are available. Then computer output will give all or most of what you need, depending on the output format of the software. For programmers of software, 'efficiency' includes speed of computation and protection against underflow and overflow for data values that are very small or very large (respectively).

I think your definition of 'efficiency' is ease of computation during an exam. Even then, exactly the best path depends on what kind of calculator or computer you are allowed to use. If the data are summarized as in the table you give, then the Answer by @V.V. (+1) seems on the right track.

I assume you already know how to use the 'Total' row to find the sample means $\bar X$ and $\bar Y$ and the the sample variances $S_X^2$ and $S_Y^2$. Then all you need is the sample covariance $$S_{XY} = \frac{1}{n-1}\left(\sum X_iY_i - n\bar X \bar Y\right) = \frac{1003 - (55)(151)/10}{9},$$ in order to get $r = \frac{S_{XY}}{S_XS_Y}.$

One often says that $r^2$ (or $R^2$or R-SQ, possibly legacy notation from days when computer terminals did not use lower-case letters) is the proportion of the total variability of Y explained by regression on X. This is due to the equation

$$S_{Y|x}^2 = \frac{n-1}{n-2}S_Y^2(1 - r^2),$$

which is equivalent to the equation $r^2 = 1 - SSE/SST$ in your question. Here $S_Y^2$ is interpreted as the 'total variability of Y', $S_{Y|x}^2$ is interpreted as the 'variability of the residuals about the regression line' or 'the variability of Y unexplained by regression', and the factor $\frac{n-1}{n-2} \approx 1$ is ignored. (If $r^2 = 1,$ then the data fit the regression line perfectly, and there is no unexplained variability in the Y's; if $r^2 \approx 0,$ then there is essentially no linear component of association between X and Y and regression is not useful for using x's to predict Y's.)

You are correct that $\hat{\sigma^2} = S_{Y|x}^2 = \frac{1}{n-2}\sum_i e_i^2$ is an unbiased estimator of $\sigma^2.$