How to prove inequality on quadratic form and orthogonal projection

Question

This is from a paper I'm reading. I don't know how to prove it.

---

• Assume that $\mathbf A$ is an $n\times n$ positive semi-definite matrix which has $k$ non-zero eigenvalues. We can assume that all positive eigenvalues are larger than some positive constants $b$ and $b'$. The remaining $n-k$ eigenvalues of $\mathbf A$ are zeros.
• Let $\mathbf P$ be the orthogonal projection onto the image of $\mathbf A$ and $\mathbf Q$ is the projection onto its kernel, so that $\mathbf P + \mathbf Q = \mathbf I$.

Question 1: The paper says 'it is easy' but I don't know how to prove it. Can anyone explain it?

As $\mathbf P (\mathbf A - b'\mathbf I)^{-1} \mathbf P \succeq \mathbf 0$ and $\mathbf P (\mathbf A - b\mathbf I)^{-1} \mathbf P \succeq \mathbf 0$, it is easy to check that $$ (b-b')\mathbf P (\mathbf A - b'\mathbf I)^{-2}\mathbf P \preceq \mathbf P (\mathbf A - b\mathbf I)^{-1} \mathbf P - \mathbf P (\mathbf A - b'\mathbf I)^{-1} \mathbf P$$

---

Question 2: The paper also mentioned this equation, which I don't know how to prove:

$$tr\left(\mathbf L^T \mathbf Q(\mathbf A - b'\mathbf I)^{-2}\mathbf Q \mathbf L\right)=\frac{||\mathbf Q\mathbf L||_F^2}{b'^2}$$

Any hints or suggestions?

Answer 1

The paper states that $b' = b - \delta$. I couldn't find what $\delta$ is and I don't feel like reading the whole paper word-by-word, but I think it's safe to assume that $\delta > 0$, so $b > b'$. In that case, $$A - b {\rm I} \preceq A - b' {\rm I},$$ so $$(A - b {\rm I})^{-1} \succeq (A - b' {\rm I})^{-1},$$ i.e., $$(A - b {\rm I})^{-1} (A - b' {\rm I})^{-1} \succeq (A - b' {\rm I})^{-2}.$$

Now, \begin{align} P (A - b {\rm I})^{-1} P &- P (A - b' {\rm I})^{-1} P = P ((A - b {\rm I})^{-1} - (A - b' {\rm I})^{-1}) P \\ &= P ((A - b {\rm I})^{-1} (A - b' {\rm I}) (A - b' {\rm I})^{-1} - (A - b {\rm I})^{-1} (A - b {\rm I}) (A - b' {\rm I})^{-1}) P \\ &= P (A - b {\rm I})^{-1} ((A - b' {\rm I}) - (A - b {\rm I})) (A - b' {\rm I})^{-1} P \\ &= P (A - b {\rm I})^{-1} (A - b' {\rm I} - A + b {\rm I}) (A - b' {\rm I})^{-1} P \\ &= (b-b') P (A - b {\rm I})^{-1} (A - b' {\rm I})^{-1} P \\ &\succeq (b-b') P(A - b' {\rm I})^{-2} P. \end{align}

As for your second question, note that $(A - b'I)^{-1} = p(A - b'I)$ for some polynomial $p$ (see Horn, Johnson, "Matrix analysis", 2nd ed., Corollary 2.4.3.4). Using the fact that $Q$ is a projector on kernel, i.e., $AQ = 0$,

\begin{align} (A - b'I)^{-1} Q &= p(A - b'I) Q = p((A - b'I) Q) = p(AQ - b'Q) = p(-b'Q) \\ &= p(-b'{\rm I})Q = -(b')^{-1} Q. \end{align} Obviously, from this we have $$(A - b'I)^{-2} Q = (A - b'I)^{-1} (A - b'I)^{-1} Q = -(b')^{-1} (A - b'I)^{-1} Q = (b')^{-2} Q.$$ Also, since $Q$ is a projector, it is symmetric, so $L^TQ = (Q^TL)^T = (QL)^T$.

Using all this, $$\mathop{\rm tr}\left( L^T Q (A - b'I)^{-2} QL \right) = (b')^{-2} \mathop{\rm tr} (L^T Q QL) = (b')^{-2} \mathop{\rm tr} ((QL)^T(QL)) = \frac{\|QL\|_F^2}{(b')^2}.$$

Answer 2

1. Since $A$ is positive definite, $A=U(D\oplus 0)U^\ast$ for some unitary matrix $U$ and some $k\times k$ positive diagonal matrix $D$. Then $P=U(I_k\oplus0)U^\ast,\ Q=U(0\oplus I_{n-k})U^\ast$ and the given inequality is equivalent to $$ (b-b')(D - b'I_k)^{-2} \preceq (D - bI_k)^{-1} - (D - b'I_k)^{-1}. $$ That is, for each nonzero eigenvalue $\lambda$ of $A$, \begin{align*} &(b-b')(\lambda - b')^{-2} \le (\lambda - b)^{-1} - (\lambda - b')^{-1}\\ \Leftrightarrow&(b-b')(\lambda-b) \le (\lambda-b')^2 - (\lambda-b')(\lambda-b)\\ \Leftrightarrow&(b-b')(\lambda-b) \le (\lambda-b')(b-b')\\ \Leftrightarrow&0 \le (b-b')^2, \end{align*} where the last inequality is obviously true.
2. Presumably $L$ is a real matrix. We have $Q=U(0\oplus I_{n-k})U^\ast$. Therefore \begin{align*} \operatorname{tr}\left(L^T Q(A - b'I)^{-2}Q L\right) &=\operatorname{tr}\left(L^T U(0\oplus b'^{-2}I_{n-k})U^\ast L\right)\\ &=b'^{-2}\operatorname{tr}\left(L^T U(0\oplus I_{n-k})U^\ast L\right)\\ &= b'^{-2}\operatorname{tr}\left(L^T Q^2 L\right)\\ &=\frac{\|QL\|_F^2}{b'^2} \end{align*}