If $A \leq B$ for a positive-definite operator $A$ in finite dimensions, then $B^{-1} \leq A^{-1}$

Question

Exercise 13 from SEC. 82 of Finite-Dimensional Vector Spaces - 2nd Edition by Paul R. Halmos.

If a linear transformation $A$ on a finite-dimensional inner product space is strictly positive (positive-definite), and if $A \leq B$, then $B^{-1} \leq A^{-1}$.

(The underlying field is not specified as real or complex.)

---

I have been able to establish the assertion in the simple case wherein $A = 1$. Unable to establish the assertion in the other case wherein $A \neq 1$. Would appreciate pointers. Thanks.

Proof for the case $A = 1$: let $[B] = (\beta_{ij})$ to be the diagonal form for $B$ under a suitable orthonormal basis $X$. Because $A = 1$, the matrix $[A] = (\alpha_{ij})$ of $A$ under $X$ is also a diagonal matrix with each diagonal entry $\alpha_{ii}$ as $1$. Now, since $B-A$ is positive, the matrix $[B]-[A]$ is positive, and therefore we have $\beta_{ii} - \alpha_{ii} \geq 0$ for all $i \implies \beta_{ii} \geq 1$ for all $i$. It follows that $\frac{1}{\beta_{ii}} \leq 1$ for all $i$, which in turn results in the diagonal matrix $[C] = \left[A^{-1}\right]-\left[B^{-1}\right]$ to have each diagonal entry positive. That is, $[C]$ is a positive matrix. Since $[C]$ together with $X$ defines the transformation $A^{-1}-B^{-1}$, we infer that $A^{-1}-B^{-1}$ is positive, and the assertion follows.

Answer 1

For $S$ symmetric, we have $SAS<SBS$ ($(ASX|SX)<(BSX|SX)$)

and we use this property with $H$ where $H$ is symmetric positive and $H^2=A^{-1}$. ( it's simple with spectral theorem) You arrive to $1<HBH$. It follows that $(HBH)^{-1}<1$, ( we use eigenvalues) and $H(HBH)^{-1}H<H^2=A^{-1}.$ And I think it's ok.

Answer 2

the idea is to use congruence transforms and square roots

$A\leq B$
$\longrightarrow I \leq A^{-\frac{1}{2}}BA^{-\frac{1}{2}}=A^{-\frac{1}{2}}B^\frac{1}{2}B^\frac{1}{2}A^{-\frac{1}{2}}$
$\longrightarrow I\leq B^\frac{1}{2}A^{-\frac{1}{2}}A^{-\frac{1}{2}}B^\frac{1}{2} = B^\frac{1}{2}A^{-1}B^\frac{1}{2}$
$\longrightarrow B^{-1}\leq A^{-1}$

recalling that for square matrices $X,Y$, then $(XY)$ has the same eigenvalues as $(YX)$

Answer 3

A verbose ("for dummies") version of the proof by user808250 is as follows.

Because $A > \textbf{0}$ and because the underlying inner product space is finite-dimensional, it is clear that $A$ is invertible. Also, $A \leq B$ means $B - A \geq \textbf{0}$. That is, for an arbitrary vector $x$, we have $\big\langle(B-A)x, x\big\rangle \geq 0$ $\implies \langle Bx, x\rangle-\langle Ax, x\rangle \geq 0$ $\implies \langle Bx, x\rangle > 0$ [$\because \langle Ax, x\rangle > 0$] $\implies B$ is strictly positive, and therefore invertible (due to finite dimensions). In summary, both $A$ and $B$ are strictly positive and invertible. This also implies that $A^\frac{1}{2}$ and $B^\frac{1}{2}$ are both strictly positive and invertible. Let $A^{-\frac{1}{2}}$ and $B^{-\frac{1}{2}}$ be the inverses of $A^\frac{1}{2}$ and $B^\frac{1}{2}$ respectively; both of $A^{-\frac{1}{2}}$ and $B^{-\frac{1}{2}}$ are strictly positive again.

Now, because the transformation $B - A$ is positive, so is $C^*(B-A)C$ for an arbitrary transformation $C$. If we let $C = A^{-\frac{1}{2}}$, then we have $C^* = {A^{-\frac{1}{2}}}^* = A^{-\frac{1}{2}}$. Substituting for $C$ and $C^*$, we have $A^{-\frac{1}{2}} (B-A) A^{-\frac{1}{2}} \geq \textbf{0}$ $\implies A^{-\frac{1}{2}}BA^{-\frac{1}{2}} - A^{-\frac{1}{2}}A^\frac{1}{2}A^\frac{1}{2}A^{-\frac{1}{2}} \geq \textbf{0}$ $ \implies A^{-\frac{1}{2}}BA^{-\frac{1}{2}} \geq \textbf{1}$.

Next, a consideration of a unitarily diagonalized form of $A^{-\frac{1}{2}}BA^{-\frac{1}{2}}$ shows that each diagonal entry (Eigenvalue) is $ \geq 1$. This implies that $A^{-\frac{1}{2}}BA^{-\frac{1}{2}}$ is strictly positive, and therefore invertible, so that each Eigenvalue of the inverse of $A^{-\frac{1}{2}}BA^{-\frac{1}{2}}$ is strictly positive and $\leq 1$. As a result, we have $\textbf{0} < \left(A^{-\frac{1}{2}}BA^{-\frac{1}{2}}\right)^{-1} \leq \textbf{1}$ $\implies \textbf{0} < A^\frac{1}{2}B^{-1}A^\frac{1}{2} \leq \textbf{1}$ $\implies \textbf{1} - A^\frac{1}{2}B^{-1}A^\frac{1}{2} \geq \textbf{0}$. It follows that $C^*\left(\textbf{1} - A^\frac{1}{2}B^{-1}A^\frac{1}{2}\right)C \geq \textbf{0}$ for an arbitrary $C$. Substituting for $C$ and $C^*$ as before, we have $A^{-\frac{1}{2}}A^{-\frac{1}{2}} - A^{-\frac{1}{2}} A^\frac{1}{2} B^{-1} A^\frac{1}{2}A^{-\frac{1}{2}} \geq \textbf{0}$ $\implies A^{-1} - B^{-1} \geq \textbf{0}$, i.e., $B^{-1} \leq A^{-1}$.