Partial ordering of positive definite matrices

Question

Assume that $\mathbf{A}$, $\mathbf{B}$, and $\mathbf{A-B}$ are all positive definite Hermitian symmetric matrices of same dimensions. Prove that $\mathbf{B}^{-1}-\mathbf{A}^{-1}$ is positive definite.

Answer 1

We use the following useful fact:

Let $\mathbf{A}$ and $\mathbf{B}$ be HPD (Hermitian positive definite). Then $$\mathbf{A}\geq \mathbf{B} \quad \Leftrightarrow \quad \mathbf{I}\geq \mathbf{A}^{-1/2}\mathbf{B}\mathbf{A}^{-1/2}.$$

(Here $\mathbf{A}\geq \mathbf{B}$ means that $\mathbf{A}-\mathbf{B}$ is positive semidefinite and $\mathbf{I}$ is the identity matrix.)

Proof: $\mathbf{A}\geq \mathbf{B}$ if (and only if) for any $\mathbf{x}$ we have $\mathbf{x}^*\mathbf{A}\mathbf{x}\geq \mathbf{x}^*\mathbf{B}\mathbf{x}$. Since $\mathbf{A}$ is HPD, any $\mathbf{x}$ can be written as $\mathbf{x}=\mathbf{A}^{-1/2}\mathbf{y}$ and hence the previous inequality is equivalent to $$\mathbf{y}^*\mathbf{A}^{-1/2}\mathbf{A}\mathbf{A}^{-1/2}\mathbf{y}=\mathbf{y}^*\mathbf{I}\mathbf{y}\geq \mathbf{y}^*\mathbf{A}^{-1/2}\mathbf{B}\mathbf{A}^{-1/2}\mathbf{y}.$$ This means that $\mathbf{I}\geq \mathbf{A}^{-1/2}\mathbf{B}\mathbf{A}^{-1/2}$. Q.E.D.

Now we use twice the useful fact and the similarity relations to prove the statement of the question. Let $\mathbf{A}$ and $\mathbf{B}$ be HPD and $\mathbf{A}\geq \mathbf{B}$. Using the useful fact we have that $\mathbf{I}-\mathbf{A}^{-1/2}\mathbf{B}\mathbf{A}^{-1/2}\geq 0$. We have the following similarity relations: $$\mathbf{I}-\mathbf{A}^{-1/2}\mathbf{B}\mathbf{A}^{-1/2}\sim \mathbf{I}-\mathbf{A}^{-1}\mathbf{B}\sim \mathbf{I}-\mathbf{B}^{1/2}\mathbf{A}^{-1}\mathbf{B}^{1/2}.$$ Hence the matrix $\mathbf{I}-\mathbf{B}^{1/2}\mathbf{A}^{-1}\mathbf{B}^{1/2}$ is semidefinite. Using again the useful fact we have that $\mathbf{B}^{-1}\geq \mathbf{A}^{-1}$, that is, $\mathbf{B}^{-1}-\mathbf{A}^{-1}\geq 0$.

You can adapt it to your special case by replacing all occurrences of "$\geq$" and "semidefinite" by, respectively, "$>$" and "definite"; except in the proof of the useful fact where "any $\mathbf{x}$" must be replaced by "any nonzero $\mathbf{x}$".

Answer 2

First, suppose that $\|A-B\|<(\|A\|^{-1})^{-1}$. Writing $C=A-B$ we find that $$B^{-1} = (A-C)^{-1}=((I-CA^{-1})A)^{-1} = A^{-1}(I-CA^{-1})^{-1}$$ which expands into Neumann series $$B^{-1} = \sum_{n=0}^\infty A^{-1}(CA^{-1})^{n}$$ Hence, $$B^{-1}-A^{-1} = \sum_{n=1}^\infty A^{-1}(CA^{-1})^{n}$$ where every term in the series, e.g., $A^{-1}CA^{-1}CA^{-1}$, is positive, being a symmetric word in two positive matrices.

The general case reduces to the above by considering the finite sequence $$A_k= A-\frac{k}{n}(A-B), \quad k=0,\dots,n$$ where $n$ is sufficiently large so that $$\frac{1}{n}\|A-B\|<\|B^{-1}\|^{-1}$$ holds. Apply the above to $A_{k-1}$ and $A_{k }$, $k=1,\dots,n$. (This is possible because the smallest eigenvalue of $A_{k-1}$ is bounded from below by the smallest eigenvalue of $B$).