Proof of $\operatorname{rank} \left( X’ V X \right) = \operatorname{rank} (X)$

Question

From page 39 of George Seber's A Matrix Handbook for Statisticians:

Let $V$ be a non-negative definite $n \times n$ matrix, and let $X$ be an $n \times p$ matrix. Then the following statements are equivalent $$ \operatorname{rank} \left( X’ V X \right) = \operatorname{rank} (X) \iff \operatorname{col} (X) \cap[\operatorname{col}(V)]^{\perp} = 0 $$ where $\operatorname{col}(A)$ denotes the column space of $A$.

---

I don't exactly understand the meaning of $\text{col}(X) \cap[\text{col}(V)]^{\perp}=0$. Does this mean $VX \neq 0$? Or are there any other meanings beyond it? Does this imply that if $VX\neq 0$, we have $\operatorname{rank}(X’VX) = \operatorname{rank}(X)$?

Answer 1

Here is a proof of the result. I assume that in this context, $V$ being "non-ngeative definite" implies that it is also symmetric (or Hermitian if we're dealing with complex matrices).

We note that $V$ necessarily has a non-negative definite square root $U = \sqrt{V}$, i.e. there exists a non-negative definite matrix $U$ such that $U^2 = V$. We note that $$ \operatorname{rank}(X'VX) = \operatorname{rank}(X'U'UX) = \operatorname{rank}([UX]'UX) = \operatorname{rank}(UX). $$ Note that $\operatorname{col}(U) = \operatorname{col}(V)$ and that the nullspace of $U$ is given by $\operatorname{col}(U)^\perp = \operatorname{col}(V)^\perp$. Let $x_1,\dots,x_r$ with $r = \operatorname{rank}(X)$ denote a maximial linearly independent collection of the columns of $X$. We note that $\operatorname{col}(X)$ is spanned by the columns $x_1,\dots,x_r$ and that $\operatorname{col}(UX)$ is spanned by the vectors $Ux_1,\dots,U x_r$.

Suppose that $\operatorname{rank}(UX) =\dim \operatorname{col}(UX) = r$. It follows that the spanning set $\{Ux_1,\dots,Ux_r\}$ must be linearly independent. Thus, for any scalars $c_1,\dots,c_r$, we have $$ U(c_1x_1 + \cdots + c_r x_r) = 0 \implies c_1 Ux_1 + \cdots + c_r Ux_r = 0 \implies c_1 = \cdots = c_r = 0. $$ That is, the only element $x = c_1 x_1 + \cdots + c_r x_r$ of the column-space satisfying $Ux = 0$ is $x = 0$. Thus, $\operatorname{col}(V)^\perp \cap \operatorname{col}(X) = \{0\}$.

Conversely, if the $\operatorname{col}(V)^\perp \cap \operatorname{col}(X) = \{0\}$, then we similarly conclude that $U(c_1x_1 + \cdots + c_r x_r) = 0 \implies c_1 = \cdots = c_n = 0$, which means that the set $\{Ux_1,\dots,Ux_r\}$ is linearly independent, which means that the spanning set $\{Ux_1,\dots,Ux_r\}$ of $\operatorname{col}(UX)$ is linearly independent, which means that $\dim \operatorname{col}(UX) = r$.