Derivative of symmetric matrix (function of symmetric matrix) w.r.t. its unique elements

486 Views Asked by At

I have an optimization problem to estimate a symmetric covariance matrix of order $q$ , say $\Sigma$, which is only formulated in terms of its unique elements (on and above/below the diagonal) that is $vech(\Sigma)=(\sigma_1,\sigma_2,...\sigma_h)$, where $h=\frac{q(q+1)}{2}$.

In the process of optimization, the first and second derivative of the log-likelihood are needed with some component such as

$\frac{\partial\,\,\text{log}|\Sigma|}{\partial\sigma_{ij}} = e_i \,\Sigma^{-1} e'_i \,\,(\text{if}\; i=j) $

$\frac{\partial\,\,\text{log}|\Sigma|}{\partial\sigma_{ij}} = e'_j \,\Sigma^{-1} e'_i \,\,(\text{if}\; i>j) $,

where $e_i$ is the $i$th column of $I_q$, here I omit the component $e'_i \,\Sigma^{-1} e_j $ from the second equation as long as I only interested in the unique elements of $\Sigma$. Is that right??, similarly derivatives of other functions such as $trace$ will be handled the same.

1

There are 1 best solutions below

5
On

No; $\log|\Sigma|$ depends on $\sigma_{ij}$ through both entries, the $ij$ entry and the $ji$ entry, since both change when $\sigma_{ij}$ changes; thus, both terms have to be included in the derivate. You can readily check this for the case $q=2$, where you can calculate the derivative and the inverse explicitly.