I was recently reviewing a research paper and came across an inequality expressed as follows: \begin{align} & \text{tr}\Big[\Big(\frac{1}{np}X^\top X + \rho B\Big)^{-1} \Big(\frac{1}{np}X^\top X\Big) \Big(\frac{1}{np}X^\top X + \rho B\Big)^{-1} \Big(\frac{1}{np}X^\top X\Big)\Big] \\ \leq & \text{tr}\Big[\Big(\frac{1}{np}X^\top X + \rho B\Big)^{-1} \Big(\frac{1}{np}X^\top X\Big)\Big] \tag{1}, \end{align} where $X_{n\times p}$ is a design matrix, $\rho>0$ is a tuning parameter, and $B_{p\times p}$ is a positive definite matrix.
From my perspective, considering that $(\frac{1}{np}X^\top X + \rho B)^{-1}$ is positive definite (PD) and $X^\top X$ is positive semidefinite (PSD), we can simplify the expression as: \begin{align*} & \text{tr}\Big[\Big(\frac{1}{np}X^\top X + \rho B\Big)^{-1} \Big(\frac{1}{np}X^\top X\Big) \Big(\frac{1}{np}X^\top X + \rho B\Big)^{-1} \Big(\frac{1}{np}X^\top X\Big)\Big] \\ \leq & \text{tr}\Big[\Big(\frac{1}{np}X^\top X + \rho B\Big)^{-1} \Big(\frac{1}{np}X^\top X\Big)\Big] \; \text{tr}\Big[\Big(\frac{1}{np}X^\top X + \rho B\Big)^{-1} \Big(\frac{1}{np}X^\top X\Big)\Big]. \end{align*} Now, if we can establish the inequality: \begin{equation} \text{tr}\Big[\Big(\frac{1}{np}X^\top X + \rho B\Big)^{-1} \Big(\frac{1}{np}X^\top X\Big)\Big] \leq 1 \tag{2}, \end{equation} we can obtain the desired result. However, this may seem counterintuitive, as proving equation (2) directly implies that the original trace equation is less than or equal to 1.
I appreciate any insights or input on this matter. Thank you!
It suffices to show $$Tr[(A+B)^{-1}A(A+B)^{-1}A] \leq Tr[(A+B)^{-1}A] $$ where $A$ is positive semi-definite and $B$ is positive definite.
Since $A\leq A+B$ we have $(A+B)^{-1/2}A(A+B)^{-1/2} \leq I_p$. Letting $C = (A+B)^{-1/2}A(A+B)^{-1/2}$, we obtain as a consequence $C^{1/2} C C^{1/2} \leq C^{1/2}C^{1/2}$, i.e., $C^2\leq C$ which rewrites
$$(A+B)^{-1/2}A(A+B)^{-1}A(A+B)^{-1/2} \leq (A+B)^{-1/2}A(A+B)^{-1/2}.$$ Taking the trace and leveraging the cyclic property on each side yields $$Tr[(A+B)^{-1}A(A+B)^{-1}A] \leq Tr[(A+B)^{-1}A].$$