How to perform these matrix computations?

Question

I was going through Boyd's Optimization text which gives the following computations,

$\mathbf{tr}(Z+tV)^{-1}$
$=\mathbf{tr}(Z^{-1}(I+tZ^{-1/2}VZ^{-1/2})^{-1})$
$=\mathbf{tr}(Z^{-1}Q(I+t\Lambda)^{-1}Q^T)$
$=\mathbf{tr}(Q^TZ^{-1}Q(I+t\Lambda)^{-1})$

where, $\mathbf{tr}$ denotes the trace; $t \in R$; $Z\succ 0$; $V \in S^n$; and $Z^{-1/2}VZ^{-1/2}=Q \Lambda Q^T$ is the eigenvalue decomposition.

I am unable to understand how these steps are performed. Can someone explain in detail how these steps proceed?

Answer 1

I think that the trickiest part of this computation is to remember that $tr(AB)=tr(BA)$, provided both are defined. Now, let's get to the derivation:

Step $1$: Rewrite

$$tr((Z+tV)^{-1})=tr((Z^{1/2}Z^{1/2}+tZ^{1/2}Z^{-1/2}VZ^{-1/2}Z^{1/2})^{-1})$$

provided $Z^{1/2}$ makes sense (as it shows up in the question, I am assuming that it makes sense). The products $Z^{1/2}Z^{-1/2}$ reduce to the identity and the product $Z^{1/2}Z^{1/2}=Z$.

Step $2$: Factor out $Z^{1/2}$ to get

$$tr((Z^{1/2}Z^{1/2}+tZ^{1/2}Z^{-1/2}VZ^{-1/2}Z^{1/2})^{-1})=tr((Z^{1/2}(I+tZ^{-1/2}VZ^{-1/2})Z^{1/2})^{-1}).$$

Step $3$: Apply $(AB)^{-1}=B^{-1}A^{-1}$ to get

$$tr((Z^{1/2}(I+tZ^{-1/2}VZ^{-1/2})Z^{1/2})^{-1})=tr(Z^{-1/2}(I+tZ^{-1/2}VZ^{-1/2})^{-1}Z^{-1/2})$$

Step $4$: Apply $tr(AB)=tr(BA)$ to move the final $Z^{-1/2}$ to the front to get

$$tr(Z^{-1/2}(I+tZ^{-1/2}VZ^{-1/2})^{-1}Z^{-1/2})=tr(Z^{-1}(I+tZ^{-1/2}VZ^{-1/2})^{-1})$$

Step $5$: Substitute $Z^{-1/2}VZ^{-1/2}$ with $Q\Lambda Q^T$ to get

$$tr(Z^{-1}(I+tZ^{-1/2}VZ^{-1/2})^{-1})=tr(Z^{-1}(I+tQ\Lambda Q^T)^{-1})$$

Step $6$: Recall that $Q^{-1}=Q^T$, so we replace $I$ by $QQ^{T}$ to get

$$tr(Z^{-1}(I+tQ\Lambda Q^T)^{-1})=tr(Z^{-1}(QQ^T+tQ\Lambda Q^T)^{-1})$$

Step $7$: Factor out the $Q$ and $Q^T$ to get

$$tr(Z^{-1}(QQ^T+tQ\Lambda Q^T)^{-1})=tr(Z^{-1}(Q(I+t\Lambda)Q^T)^{-1})$$

Step $8$: Apply $(AB)^{-1}=B^{-1}A^{-1}$ to get

$$tr(Z^{-1}(Q(I+t\Lambda)Q^T)^{-1})=tr(Z^{-1}(Q^T)^{-1}(I+t\Lambda)^{-1}Q^{-1})$$

Step $9$: Since $Q^{-1}=Q^T$, we can simplify this to

$$tr(Z^{-1}Q^{-1}(I+t\Lambda)^{-1}(Q^T)^{-1})=tr(Z^{-1}Q(I+t\Lambda)^{-1}Q^T)$$

Step $10$: Since $tr(AB)=tr(BA)$, we finally get

$$tr(Z^{-1}Q(I+t\Lambda)^{-1}Q^T)=tr(Q^TZ^{-1}Q(I+t\Lambda)^{-1})$$