Derivation of the trace with Hessian matrix

1.2k Views Asked by At

Take the matrix $\Sigma\in\mathbb{R}^{n\times n}$ and the function $f:\mathbb{R^n}\rightarrow\mathbb{R}$ in $C^2$.

1) How can I compute the matrix derivation

$$\frac{\partial (tr\left(\Sigma\Sigma^TD^2f\right))}{\partial{\Sigma}}$$

where $tr$ is the trace operator and $D^2f$ the Hessian matrix?

2) How does it simplify, if I choose $\Sigma$ to be a diagonal matrix?

My aim is to see if there can be found a form of the derivative in terms of the Laplacian $\Delta{f}$ and the gradient $\nabla{f}$.

1

There are 1 best solutions below

1
On

You may be able to use a standard result regarding matrix derivatives:

$$ \frac{\partial Tr(X^TAX)}{\partial X} = (A + A^T)X $$ Separately, the Trace has the property (for appropriate matrix dimensions):

$$ Tr(AB) = Tr(BA) $$

Now, denoting the Hessian of $f$ as $H$, We obtain:

$$ \frac{\partial Tr(\Sigma \Sigma^T H)}{\partial \Sigma } = \frac{\partial Tr(\Sigma^T H\Sigma )}{\partial \Sigma } = (H + H^T)\Sigma = 2H\Sigma $$

Where the last equality follows since the Hessian is symmetric.

For the second part of your question, it should be fairly obvious that if $\Sigma$ is diagonal, then $2H\Sigma $ becomes the Hessian $H$ with each column $h_i$ scaled by a factor of $2\Sigma_{ii}$.