Linear regression: how does multicollinearity inflate variance of estimators

Question

Suppose I have a multiple linear regression model $Y_i = \beta_0 + \beta_1 * X_{i1} + ... \beta_p * X_{ip} + \epsilon_i$ where $\epsilon_i \sim N(0, \sigma^2)$ and $Cov(\epsilon_i, \epsilon_j) = 0$ for $i \neq j$. If two of the predictors are correlated, then that will inflate the variance of the coefficient estimates, leading to invalid inferences. In the MLR setting, the variance of the least squares estimator $\hat{\beta}$ is $\sigma^2(X'X)^{-1}$. Can someone give me some mathematical intuition as to why $\sigma^2(X'X)^{-1}$ will inflate/become unstable if there's multicollinearity?

Answer 1

It's easiest to see this in terms of the eigenvalues of $X'X$. Since $x'X$ is symmetric and real, its eigenvalues are real, and we can diagonalize it as

$X'X=U\Lambda U'$

where $\Lambda$ is a diagonal matrix of the eigenvalues of $X'X$, and $U$ is an orthogonal matrix. Then

$(X'X)^{-1}=U \Lambda^{-1}U'$

If there is multicolinearity, then $X'X$ is nearly singular, or in terms of the eigenvalues, one or more of the eigenvalues is very small or zero. Very small eigenvalues cause $\Lambda^{-1}$ to have very large entries. This in turn causes $(X'X)^{-1}$ to have very large entries.