Derivatives of a trace with respect to perturbation

Question

Let $\mathbf{X}$ and $\mathbf{Y}$ be symmetric positive-definite $n\times n$ matrices. Let $\{\lambda_i(\mathbf{A})\}$ denote the eigenvalues of $n\times n$ matrix $\mathbf{A}$ and $\mathbf{\Lambda}_{\mathbf{A}}$ denote the corresponding diagonalizing unitary matrix (if it exists) such that $\mathbf{\Lambda}_{\mathbf{A}}^T\mathbf{A}\mathbf{\Lambda}_{\mathbf{A}}=\mathbf{D}_{\mathbf{A}}$ where diagonal matrix $\mathbf{D}_{\mathbf{A}}$ contains $\{\lambda_i(\mathbf{A})\}$.

Let $\mathbf{Z}=\mathbf{X}+a\mathbf{Y}$ where $a>0$ is a scalar. Clearly, $\mathbf{Z}$ is also symmetric positive-definite. By the usual definition of the application of functions to matrices, $\mathbf{Z}\log \mathbf{Z}=\mathbf{Z}\mathbf{\Lambda}_{\mathbf{Z}}\log(\mathbf{D}_{\mathbf{Z}})\mathbf{\Lambda}_{\mathbf{Z}}^T$, where $\log(\mathbf{D}_{\mathbf{Z}})$ contains logs of diagonal elements. Consequently, trace $$T=\operatorname{Tr}[\mathbf{Z}\log \mathbf{Z}]=\sum_{i=1}^n\lambda_i(\mathbf{Z})\log \lambda_i(\mathbf{Z}).$$ I am interested in the derivative of the trace with respect to $a$. Is it possible to write expressions for $\frac{\partial T}{\partial a}$, $\frac{\partial^2 T}{\partial a^2}$, and $\frac{\partial^3 T}{\partial a^3}$ in terms of $a$ and eigenvalues $\{\lambda_i(\mathbf{X})\}$ and $\{\lambda_i(\mathbf{Y})\}$ of $\mathbf{X}$ and $\mathbf{Y}$ (as well as $\mathbf{\Lambda}_{\mathbf{X}}$ and $\mathbf{\Lambda}_{\mathbf{Y}}$)?

Answer 1

The answer to your question is "yes," but I guess you wanted to see what the expressions would be!

The Matrix Cookbook is your friend for this type of question. http://www.math.uwaterloo.ca/~hwolkowi/matrixcookbook.pdf
You can use the equations in Section 2.3.

First, by the product rule: $$ \frac{\partial T}{\partial a} = \sum_{i=1}^n \left[ \frac{\partial \lambda_i(\mathbf{Z})}{\partial a} \log \lambda_i(\mathbf{Z}) + \lambda_i(\mathbf{Z}) \frac{\partial \log \lambda_i(\mathbf{Z})}{\partial a} \right] $$

Next note that $$ \frac{\partial \log \lambda_i(\mathbf{Z})}{\partial a} = \frac{1}{\lambda_i(\mathbf{Z})} \frac{\partial \lambda_i(\mathbf{Z})}{\partial a} $$ so the only "tricky" thing is to compute the partial of the eigenvalues with respect to $a$. That's basically Eqn. 67 in the Matrix Cookbook: $$ \frac{\partial \lambda_i(\mathbf{Z})}{\partial a} = \mathbf{v_i}^T \frac{\partial \mathbf{Z}}{\partial a} \mathbf{v}_i $$ where $\mathbf{v}_i$ are the eigenvectors of $\mathbf{Z}$, i.e. are columns of $\mathbf{\Lambda}_\mathbf{Z}$. Finally $$ \frac{\partial \mathbf{Z}}{\partial a} = \mathbf{Y} $$.

You are on your own to combine the terms and to compute the higher derivatives from here. (An exercise for the reader.)

Answer 2

Here is a partial anwser.

In Section 2.5 of the Matrix Cookbook, there is an unnumbered formula between Eqn 98 and Eqn 99 which yields the derivative for the trace of $any$ matrix function.

A few iterations with that formula should convince you that the $k$-th derivative of your function as $$ (-1)^k \,{\rm tr}\big(Y^k Z^{1-k}\big) $$ for $k>1$.

For $k=1,\,$ the result includes a $\log$-term $$ {\rm tr}\big(Y\log(Z)+Y\big) $$ The complete answer involves reformulating these results in terms of the eigenvalues.