Why are the OLS residuals and $\hat{\beta}$ uncorrelated?

1.6k Views Asked by At

I'm trying to show that $\hat{e}$ and $\hat{\beta}$, the residuals and coefficient vector for the OLS problem $y=X\beta+\epsilon$, are uncorrelated.

The proof I see online doesn't make sense to me because it doesn't subtract $E[\hat{e}]E[\hat{\beta}]$ when computing the covariance.

From page 21 of http://homepage.ntu.edu.tw/~ckuan/pdf/et01/et_Ch3.pdf it says $$ cov(\hat{e},\hat{\beta}) = E[(I-P)yy'X(X'X)^{-1}] \\ = (I-P)E[yy']X(X'X)^{-1} = \sigma_0^2(I-P)X(X'X)^{-1} = 0. $$ where $P=X(X'X)^{-1}X'$ is the orthogonal projection onto $span(X)$. This seems wrong to me because $cov(A,B)=E[AB^{T}]-E[A]E[B^T]$, but above is just $E[\hat{e}\hat{\beta}^T].$

1

There are 1 best solutions below

3
On BEST ANSWER

The vector $\hat e$ of residuals has expectation zero. Here's the proof: By definition, $$\hat e:=y-\hat y=y-X\hat \beta_T.$$ Recall Theorem 3.4(a) stated that $$ E(\hat\beta_T)=\beta_0 $$ where $\beta_0$ is such that $E(y)=X\beta_0$, as specified in Classical Condition [A2] (section 3.2.1). Since $X$ is non-stochastic (condition [A1]), it follows that $$ E(X\hat\beta_T)=XE(\hat\beta_T)=X\beta_0 = E(y). $$ Or you can prove this directly using the definition $\hat\beta_T:=(X'X)^{-1}X'y$: $$E(X\hat\beta_T)=E(X(X'X)^{-1}X'y)\stackrel{[A1]}=X(X'X)^{-1}X'E(y)\stackrel{[A2]}=X(X'X)^{-1}X'X\beta_0=X\beta_0=E(y).$$