Let $A, B, W \in \mathbb{R}^{n \times n}$ be three positive definite matrices, and assume $A - B$ is positive definite as well. Let $\{\lambda_{i}\}_{i=1}^{n}$ and $\{u_{i}\}_{i=1}^{n}$ be the eigenvlaues of matrices $A$ and $B$, respectively, sorted in decreasing order.
I wish to prove the three properties on top of page 109 of the convex optimization textbook:
- $tr(WA) \geq tr(WB)$,
- $tr(A^{-1}) = \sum_{i=1}^{n}\frac{1}{\lambda_i} \leq \sum_{i=1}^{n}\frac{1}{u_i} = tr(B^{-1})$,
- $det(A) = \Pi_{i=1}^{n} \lambda_i \geq \Pi_{i=1}^{n} u_i\geq det(B)$.
I have no idea how to prove 1. For 2 and 3, I feel we need to prove $\lambda_i \geq u_i$ for any $i \leq n$? Any ideas?
Property 1
First note that the trace is invariant under cyclic permutations and so $$ \mathrm{Tr}[WA] = \mathrm{Tr}[W^{1/2}A W^{1/2}]. $$ Secondly note that if $X > 0$ then $Y^T X Y \geq 0$ for any square matrix $Y$ acting on $\mathbb{R}^n$. Thus we have $W^{1/2} (A-B) W^{1/2} \geq 0$. Therefore as the trace of a positive semidefinite matrix is positive we get $$ \begin{aligned} \mathrm{Tr}[WA] - \mathrm{Tr}[WB] &= \mathrm{Tr}[W(A-B)] \\ &= \mathrm{Tr}[W^{1/2}(A-B)W^{1/2}] \\ &\geq 0 \end{aligned} $$
Property 2
The equalities in property 2 follow from the fact that the trace is equal to the sum of the eigenvalues and that the eigenvalues of $X^{-1}$ are the inverses of the eigenvalues of $X$ (where $X$ is a positive definite matrix). For the inequality, if $A > B$ then $A^{-1} < B^{-1}$, you can find a proof of this statement here.
Property 3
This has been answered before here. Essentially you can use the identity that $A > B > 0 \iff B^{-1/2}BB^{-1/2} > B^{-1/2}AB^{-1/2} > 0$ together with the multiplicative property of determinants and the fact that the determinant of a positive matrix is positive.