Prove that $\text{rank}(X^TX)=\text{rank}(X)$

Question

Prove, for real $X$, that $\text{rank}(X^TX)=\text{rank}(X)$.

Could anyone please help me with this problem? I've tried to use full-rank factorization and rank-related theorems mentioned in my book but still failed to solve this. I am learning linear algebra by myself and my book has no solutions manual so I find it really hard to get used to solve linear algebra problems.

Many thanks in advance for your help!

Answer 1

Assume $u$ is in the null space of $\mathbf{X}^T\mathbf{X}$.
Then $\mathbf{X}^T\mathbf{X}u=0 \implies u^T\mathbf{X}^T\mathbf{X}u = u^T0=0$.

Then assume, $u$ is not in the null space of $\mathbf{X}$.
Then $\mathbf{X}u=v$, (notice $u^T\mathbf{X}^T = v^T$).

But $u^T\mathbf{X}^T\mathbf{X}u = 0 \implies v^Tv=0$, which implies $v=\mathbf{X}u=0$.

Thus, $null(\mathbf{X}^T\mathbf{X})\subset null(\mathbf{X})$.

Proving the reverse inclusion is trivial.

So, $null(\mathbf{X}^T\mathbf{X}) = null(\mathbf{X})$.

By the rank and nullity theorem (Nullity of A + Rank of A = Total number of columns in A),
rank$(\mathbf{X}^T\mathbf{X})$ = rank$(\mathbf{X})$.

Answer 2

Consider the map $T: Im(X)\to Im(X^TX)$ given by $T(w)=X^Tw$. We show that $T$ is an isomorphism.

$T$ is obviously onto, since given any $v\in Im(X^TX)$, we have $v=X^TXv'=X^T(Xv')=T(Xv')$ for some $v'$.

It remains to check that $T$ is one-to-one.

Suppose $T(w)=0$. We have $w=Xv$ for some $v$, so $X^TXv=0$. In particular, $(X^TXv, v)=0$. Thus $(Xv, Xv)=0$, so by positivity o the inner product, we conclude that $Xv=w=0$ as desired.

Answer 3

Lord KK above provided a sufficient answer. I will provide another approach which requires the knowledge of spectral decomposition, singular value decomposition (SVD) and the following lemma.

Lemma: For a full rank matrix $\mathbf{A}_{s\times s}$ and any $\mathbf{B}_{s\times t}$, $rank(\mathbf{A}\mathbf{B}) = rank(\mathbf{B})$.

$$\begin{aligned} rank(\mathbf{X}^T\mathbf{X}) & = \text{number of non-zero eigenvalues of }\mathbf{X}^T\mathbf{X} \\ & = \text{number of non-zero singular values of }\mathbf{X} \\ & \overset{*}{=} rank(\mathbf{X}), \end{aligned}$$

where the final equality follows from the definitions of SVD and the lemma, $$\begin{aligned} rank(\mathbf{X}) & = rank(\mathbf{U} \mathbf{D} \mathbf{V}^T) \\ & = rank( \mathbf{D} \mathbf{V}^T) \ \ \ \ (\text{Lemma})\\ & = rank( \mathbf{D}) \ \ \ \ \ \ \ \ \ \ (\text{Lemma})\\ & = \text{number of non-zero singular values of }\mathbf{X} \\ \end{aligned}$$