What is the estimate coefficient when design matrix is singular, when using the Moore-Penrose generalized inverse matrix?

Question

The OLS estimators is

But if $Y$ is a $50 \times 1$ matrix, and the design matrix $X$ is $50 \times 16$.

The design matrix $X^T X$ is singular, then what will the estimate coefficient function be, when using the Moore-Penrose generalized inverse matrix?

Is the function still the same, but just have to replace the inverse matrix of $X^T X$ into the Moore-Penrose generalized inverse matrix of $X^T X$?

Thank you very much.

Answer 1

The case when $X^T X$ is singular is usually called multicollinearity. There are several standart ways to deal with it:

1. You can just get rid of the strongly correlated features.

2. You can use regularization ($l_{1}$ or $l_{2}$ is a common choice: in case of $l_{2}$ regularization the formula for betas will be: $\hat \beta = (X^TX + \lambda I)^{-1}X^T y, \ $ where $I$ is identity matrix.)

3. You can use principal components, as they are always uncorrelated.