Correlation Coefficients in Simple Linear Regression Model

Question

I need to compute the correlation between $y$ and $\hat y$, between $\hat y$ and $r$, and between $y$ and $r$. In this case, $\hat y$ is the estimator of $y$, and $r$ is the residual. The catch is that I need these in terms of $S_{xx}$, $S_{yy}$, and $S_{xy}$, but I only know how to do it with the hat matrix $H$. I have that the correlations in terms of that are $\sqrt H$, $0$, and $\sqrt {I-H}$ respectively, where $I$ is the identity matrix. Any help would be greatly appreciated!

Answer 1

if i understood correctly, you need these formulas; Suppose r ; $r_1,r_2,...,r_n$ and y; $y_1,y_2,...,y_n$

$$s_{rr}=\sum_{i=1}^n \frac{(r_i-\overline r)^2}{n-1}$$ $$s_{yy}=\sum_{i=1}^n \frac{(y_i-\overline y)^2}{n-1}$$ $$s_{ry}= \sum_{i=1}^n \frac{(r_i-\overline r)(y_i-\overline y)}{n-1}$$ $$correlation=\frac {s_{ry}}{\sqrt {s_{rr}s_{yy}}}$$

Answer 2

Partial answer/hints:

1. The correlation coefficient between $r$ and $\hat{y}$ is $0$ as they are orthogonal.

2. The correlation coefficient between $r$ and $y$ is $0$ due the first order condition in the OLS derivation.

3. The correlation coefficient between $y$ and $\hat{y}$ is the correlation coefficient between $y$ and $x$, namely, $$ r_{y,\hat{y}}=r_{x,y}=\frac{S_{xy}}{S_xS_y} \, . $$