If $A^TA$ is invertible and $W$ is square what is $\operatorname{rank}A^TWA$?

Question

Consider $A ∈ \mathbb{R}^{m \times n}$ and $W ∈ \mathbb{R}^{m \times m}$ such that $n < m$ and $A^TA$ is invertible. If we know that $ \operatorname{rank}W = w \ge n $ what can we say about $\operatorname{rank}(A^TWA)$? My intuition is $ n + w - m \le \operatorname{rank}(A^TWA)$ but I can't prove it. I think using $ \operatorname{rank}(A^TWA) \ge \operatorname{rank}(A^TW) + \operatorname{rank}A - m $ may be helpful. Thanks!

Answer 1

Your proposed formula can't be true. Consider $n=1, m=2$ and $A= \begin{pmatrix} 1\\ 0\end{pmatrix}$. Then $A^TA=(1)$ is an invertible $1 \times 1$-matrix. With $W= \begin{pmatrix} 0& 1\\ 1&0 \end{pmatrix}$ we have $\mathrm{rank}(W)=w =2 >n =1$. However, $A^TWA = (0)$ has rank zero. Since $n+w-m = 1+2-2=1$ in this case, the inequality $n+w-m \le \mathrm{rank}(A^TWA)$ can't be true.

In general it is true that $\mathrm{rank}(WA) \ge n+w-m$ since the image of $A$ contains $n$ linearly independent vectors of which at most $m-w$ can be send to $0$ or to linearly dependent images, so the rank has a lower bound of $n-(m-w) = n+w-m$. However, if we now apply $A^T$, it is possible that all the vectors in the image of $WA$ are in the kernel of $A^T$ since $A^T: \mathbb R^m \to \mathbb R^n$ maps into a lower-dimensional space, so it has $m-n > 0$ linearly independent directions mapping to $0$. Since at most $m-n$ vectors can be mapped to $0$, we have $n+w-m -(m-n) = 2n+w-2m$ as a lower bound for $\mathrm{rank}(A^TWA)$. Note that this number can be negative, depending on $n,m$ and $w$.