Proof that $X^TX$ is not invertible

Question

What scenarios would cause $X^TX$ to not be invertible?

Answer 1

Since $n>m$, there is a non-zero vector $v$ such that $Xv=0$, and then $X^TXv=0$ as well. Hence $X^TX$ cannot be invertible.

Answer 2

You should be able to prove that for any matrices $A,B$ such that $AB$ is defined, rank(AB) $\leq$ min(rank(A),rank(B)) from the definition of rank. See this question and the duplicates therein.

Now, take $A = X^T, B= X$. The rank of $A,B$ is $m$ (since it is full rank). So, $AB$ is a n x n matrix with rank at most $m$ which is less than $n$ so $AB$ is not invertible.

Note that the fact that $X$ is full rank is irrelevant. The only thing we're relying on is $m<n$.

Answer 3

The answers given are absolutely correct. Let me add my voice to the chorus. This is true if your $X$ is a square matrix.

A Matrix is singular (not invertible) if and only if its determinant is null.

By the properties of the determinant: $$\det{(A)} = \det{(A^T)}$$ And by Binet's theorem: $$\det{(A \cdot B)}=\det{(A)} \det{(B)}$$ Then, you're requesting that: $$\det{(X^T X)} = 0$$ $$\det{(X^T)} \det{(X)} = \det{(X)}^2 = 0$$ $$ \implies \det{(X)} = 0$$ That means you get a singular matrix only by using a singular matrix.