Ordering of positive semidefinite matrices

Question

Suppose $A \in \mathbb R^{n \times r}$ with $r < n$ and $M, X \in \mathbb R^{n \times n}$ are symmetric and positive definite. I am wondering whether the following relation is correct: \begin{align*} A(M + A^{\top}XA)^{-1} A^{\top} < X^{-1}, \end{align*} where $<$ means the Loewner ordering.

Answer 1

Your relation is indeed correct.

Note that \begin{align*} A(M + A^{T}XA)^{-1} A^{T} < X^{-1} & \iff\\ X^{1/2}A(M + A^{T}XA)^{-1} A^{T}X^{1/2} < X^{1/2}X^{-1}X^{1/2} & \iff\\ (X^{1/2}A)(M + (X^{1/2}A)^T(X^{1/2}A))^{-1} (X^{1/2}A)^T < I. \end{align*} Now, if we define $N = X^{1/2}A$, this inequality becomes $$ N(M + N^TN)^{-1}N^T < I $$ Note that \begin{align*} N(M + N^TN)^{-1}N^T < I & \iff\\ N(M^{1/2}[I + M^{-1/2}N^T(M^{-1/2}N^T)^T]M^{1/2})^{-1}N^T < I & \iff\\ (M^{-1/2}N^T)^T(I + M^{-1/2}N^T(M^{-1/2}N^T)^T)^{-1}M^{-1/2}N^T < I.\\ \end{align*} Now, define $P = M^{-1/2}N^T$. The inequality is then $$ P^T(I + PP^T)^{-1}P < I. $$ It therefore suffices to prove the above (which is to say that it suffices to consider the $X = M = I$ case).

By this post, we can rewrite $$ P^T(I + PP^T)^{-1}P = (P^TP)(I + P^TP)^{-1}. $$ Thus, for any eigenvalue $x$ of $P^TP$, $\frac{x}{1+x}$ is an eigenvalue of $P^T(I + PP^T)^{-1}P$. Since $P^TP$ is positive semidefinite, we have $x \geq 0$. It follows that the eigenvalues of $P^T(I + PP^T)^{-1}P$ are smaller than $1$, so that we indeed have $P^T(I + PP^T)^{-1}P < I$ as desired.