Rank of product of matrices consisting of orthonormal column vectors

69 Views Asked by At

I have a matrix $\mathbf X$ that is made up of $m$ column vectors in an $n$-dimensional vector space $\mathbb{R}^n$, with $m<n$. The column vectors form a set of $m$ orthonormal vectors. So $\mathbf X$ is an $n \times m$ matrix. The rank of $\mathbf X$ is $m$.

I have another matrix $\mathbf A$ that is an $n \times n$ symmetric matrix. The rank of $\mathbf A$ is $r$.

Is it true that the product $\mathbf{X}^\intercal\mathbf{AX}$ has rank that is exactly equal to $\min(r,m)$? I suspect this may be true but cannot prove it. If it is not true, what matrices provide a counterexample?

1

There are 1 best solutions below

11
On BEST ANSWER

This is definitvely not true. As a counterexample, consider $$ \mathbf X = \pmatrix{1\\1}, \qquad \mathbf A = \pmatrix{1&0\\0&-1}. $$


One result that is illuminating here is the Cauchy interlacing theorem. If $\alpha_1 \leq \cdots \leq \alpha_n$ denote the eigenvalues of $\mathbf A$ and $\beta_1 \leq \cdots \leq \beta_m$ the eigenvalues of $\mathbf X^T \mathbf A \mathbf X$, then we have $$ \alpha_j \leq \beta_j \leq \alpha_{n-m+j}. $$ The rank of a symmetric matrix is the number of non-zero eigenvalues that it has. If $\alpha_i$ is the largest negative eigenvalue of $A$, then we can guarantee that $\beta_j$ is negative for all $j \leq i - (n-m)$. If $\alpha_i$ is the smallest positive eigenvalue of $A$, then we can guarantee that $\beta_j$ is positive for all $j \geq i$.

Putting all this together, if $\mathbf A$ has $k_-$ negative eigenvalues and $k_+$ positive eigenvalues, then $B$ has at least $\max\{0,k_- - (n-m)\}$ negative eigenvalues and $\max\{0,k_+ - (n-m)\}$. So, we have $$ \operatorname{rank}(\mathbf X^\top \mathbf A \mathbf X) \geq \max\{0,k_- - (n-m)\} + \max\{0,k_+ - (n-m)\}. $$ For all symmetric $\mathbf A$, there exists an $\mathbf X$ (with orthonormal columns) such that this lower bound is attained; that is, the inequality is tight. This is a consequence of the converse to Cauchy's interlacing theorem; one reference for this converse to the interlacing theorem is Matrix Analysis by Bhatia (Theorem III.1.9).

You might find the following to be an interesting exercise: show that this lower bound is equal to $\min\{r,m\}$ if and only if either all eigenvalues of $\mathbf A$ are positive or all eigenvalues of $\mathbf A$ are negative.

We can weaken this bound slightly to get \begin{align} \operatorname{rank}(\mathbf X^\top \mathbf A \mathbf X) &\geq \max\{0,k_- - (n-m)\} + \max\{0,k_+ - (n-m)\} \\ & \geq [k_- - (n-m)] + [k_+ - (n-m)] \\ & = (k_- + k_+) - 2(n-m) = \operatorname{rank}(A) - 2(n-m). \end{align} This simplified lower bound is the same as the other in the case that both $k_-,k_+$ are both greater than $n-m$.