Ordinary Least Squares with right hand size all zeros

Question

I have a system of equations $X\beta = 0$. The standard Ordinary Least Squares solution to $y = X\beta$:

$$ \hat\beta = (X^TX)^{-1}X^Ty $$

then only provides the trivial $\hat\beta = 0$ solution.

How can I compute a non-trivial solution?

Answer 1

First, note that the solutions $\beta$ to $X\beta=0$ are organized in the null space of $X$. Furthermore, the dimension of $\operatorname{Null}(X)$ is $n-\operatorname{rank}(X)$ where $n$ is the number of columns of $X$. This means that $X\beta=0$ has no nontrivial solutions if and only if $X$ has full column rank.

Next, matrix $X^\intercal X$ is called the Gramian of $X$. One important property of the Gramian is that is has the same rank a $X$, so $\operatorname{rank}(X^\intercal X)=\operatorname{rank}(X)$. This means that $X^\intercal X$ is nonsingular if and only if $X$ has full column rank.

Putting this together, we see that $X^\intercal X\hat{\beta}=0$ has nontrivial solutions precisely when $X\beta=0$ has nontrivial solutions. We only have $\hat{\beta}=(X^\intercal X)^{-1}X^\intercal y$ when $X$ has full column rank.