Suppose $\mathbf A$ is an $n \times r$ matrix and $\mathbf D$ a diagonal $r \times r$ matrix with all elements in the diagonal strictly positive. Also $\mathbf A\mathbf D\mathbf A'$ is invertible. I need to show that
$$ \mathbf D^{-1}-\mathbf A'(\mathbf A\mathbf D\mathbf A')^{-1}\mathbf A $$
is positive semi-definite. I am not sure where to start from though. Any hints would be very welcome.
On
Thoughts
We know that $A$ is of full (row) rank since $ADA'$ is invertible. So, there exists a $B$ with $AB = I_{r \times r}$.
This manipulation of the inequality
$$ x'\mathbf D^{-1}x-x'\mathbf A'(\mathbf A\mathbf D\mathbf A')^{-1}\mathbf Ax \geq 0 \iff \\ x'\mathbf D^{-1}x\geq x'\mathbf A'(\mathbf A\mathbf D\mathbf A')^{-1}\mathbf Ax \iff \\ x'\mathbf D^{-1}x\geq (\mathbf A x)'(\mathbf A\mathbf D\mathbf A')^{-1}(\mathbf Ax) \iff\\ \frac{(\mathbf A x)'(\mathbf A\mathbf D\mathbf A')^{-1}(\mathbf Ax)}{x'\mathbf D^{-1}x} \leq 1 \\ $$
Per hardmath's suggestion: Let $y = D^{-1/2}x$, and let $B = AD^{1/2}$. We can rewrite $$ D^{-1} - A'(ADA')^{-1}A = \\ D^{-1/2}(I - D^{1/2}A'(AD^{1/2}D^{1/2}A')^{-1}AD^{1/2})D^{-1/2} =\\ D^{-1/2}(I - B'(BB')^{-1}B)D^{-1/2} $$ To show that $I - B'(BB')^{-1}B$ is positive definite, it suffices to note that $B'(BB')^{-1}B$ defines the orthogonal projection onto the row-space of $B$. In particular, $B'(BB')^{-1}B y$ produces $B'x$, where $x$ solves the "least squares" equation $BB'x = By$. So, $I - B'(BB')^{-1}B$ defines the projection onto the orthogonal complement of the row-space.
For a more computational ending: $M = B'(BB')^{-1}B$ is symmetric with $M^2 = M$. So, its eigenvalues are in $\{0,1\}$. So, the eigenvalues of $I - M$ are also in $\{0,1\}$. So, $I - M$ is symmetric with non-negative eigenvalues.