why is E(XX') the covariance matrix

Question

In regression people often refer to the $X'X$ term as the estimate of $E(xx')$ which makes sense by LLN. However, people also refer to $E(xx')$ as the covariance matrix of $x$ which only seems to make sense if $E(x)=0$. Why does it makes sense if the $x$'s are not demeaned?

Answer 1

You are right, the covariance of a random matrix $X$ is given by $E[(X-EX)(X-EX)']$, however note that in a context of regression analysis we are interested, inter alia, in the variance of the coefficients estimators. Consider a model $Y=X\beta + \epsilon$, with $var(\epsilon) = \sigma^2I$. \begin{align} var(\hat{\beta}|X)&=var((X'X)^{-1}X'y|X)\\ &= (X'X)^{-1}X'var(y|X)X(X'X)^{-1}\\ &= (X'X)^{-1}X'E(\epsilon\epsilon')X(X'X)^{-1}\\ & = \sigma^2(X'X)^{-1}XX(X'X)^{-1}=\sigma^2 (X'X)^{-1}. \end{align}