Coefficient of Determination and Correlation between observed and fitted value in Multiple Linear Regression.

Question

Consider Multiple Linear Model $$y= X\beta + \epsilon$$ Then using Ordinary Least Square, we get estimate of $\beta$ as $$\hat{\beta} = (X'X)^{-1}X'y$$ And $$\hat{y} = X\hat{\beta}$$ $$SS_{\rm Res}= (y-X\hat{\beta})'(y-X\hat{\beta})$$ $$R^2= 1-\frac{SS_{\rm Res}}{SS_{\rm Total}} $$ From here how can we show $$R^2 = (\operatorname{Correlation} ( y, \hat{y}))^2$$ This can be easily shown in Simple Linear Model as there is only One explanatory variable but I am not able to show this in Multiple Linear Model.

Answer 1

Note that the residual $e$ is orthogonal (independent) to $\hat{y}$, moreover note that $y = \hat{y} + e$ and that $\operatorname{cov}(x,x) = \sigma^2_x$. Next, recall that $R^2$ is defined as $$ R^2 = \frac{\sum( \hat{y}_i - \bar{y})^2}{\sum( y_i - \bar{y})^2 } = \frac{ \sum( \hat{y}_i - \bar{y})^2/n }{ \sum( y_i - \bar{y})^2/n } = \frac{ \hat{\sigma}^2_{ \hat{y} } }{\sigma^2_y}, $$ hence

\begin{align} \rho ^ 2_{\hat{y}, y} &= \left( \frac{\operatorname{cov}(\hat{y},y)}{ \sigma_{\hat{y}} \sigma_y} \right) ^2\\ &= \left( \frac{\operatorname{cov}(\hat{y}, \hat{y}+e)}{\sigma_{\hat{y}} \sigma_y} \right)^2 \\ &= \left( \frac{ \sigma_{\hat{y}} ^2 }{\sigma_{\hat{y}} \sigma_y} \right)^2\\ & = \frac{ \sigma^2_{\hat{y}}}{ \sigma^2_y}\\& = R^2. \end{align}