Why is it important for the $ Z' Z$ matrix in a multivariate regression to be linear independent?

Question

I need help from the experts regarding this question:

Why is it important for the $ Z' Z$ matrix in a multivariate regression to be linear independent?

Any advice will be greatly appreciated.

Answer 1

Note that for the least squares problem has a unique solution, i.e., a unique vector of regression coefficients $\hat{\beta}$, you need $Z'Z$ to be invertible. Namely, recall that the ordinary least squares estimators for $Y = Z\beta + \epsilon$ are given by $$ \hat{\beta} = (Z'Z)^{-1}Z'Y, $$ hence, such a vectors is unique iff $(Z'Z)^{-1}$ exists and unique, and $Z'Z$ is invertible iff $Z'Z$ is a full rank matrix.