I'm trying to make sure I understand the ins and outs of a linear regression and am making a table for what is correlated with what, so just want to see if I have everything included. I'm looking at both the regression $y = X\beta + \epsilon$ and the estimated values $\hat{y} = X\hat{\beta} + \hat{\epsilon}$
- For $y$: could be correlated with $X$'s (the predictor variables), the slopes $\beta$, their estimates $\hat{\beta}$, and with the residuals $\hat{\epsilon}$
- For $\hat{y}$: same as for $y$
- For $X$: could be correlated with $y$, the \beta coefficients, $\hat{\beta}$, but not with $\epsilon$ or $\hat{\epsilon}$
- For $\beta$ and $\hat{\beta}$: Am I incorrect in assuming they could be correlated with everything?...
- $\epsilon$: could be correlated with $\beta$ or $\hat{\beta}$, but aren't disturbances (the "true error") by design not supposed to correlate with the $X$'s or the $y$'s?
- $\hat{\epsilon}$: the residuals could be correlated with the $\beta$ and $\hat{\beta}$, I think, and they could correlate with $y$ and $\hat{y}$, but residuals are not supposed to correlate with $X$'s.
Could someone point out the parts to me where I've gone wrong? Just trying to make sure I understand the linear model inside and out. Thank you!