My question is as follows:
What is the result of
$\frac{d}{d{\bf X}}{\bf X}^{-1}{\bf C}{\bf X}^{-1}$, where ${\bf X}$ is positive definite (PD), ${\bf C}$ is positive semidefinite (PSD)?
(If possible, please show me a compact result, maybe using Kronecker product)
Thanks!
Let $$F=X^{-1}CX^{-1}$$ Taking the gradient of this matrix with respect to the matrix $X$ results in a 4th order tensor.
The special (isotropic) 4th order tensor ${\mathbb E}$ with components $${\mathbb E}_{ijkl}=\delta_{ik}\,\delta_{jl}$$ can be used in a compact expression for this gradient.
Finding the differential and gradient is straightforward $$\eqalign{ dF &= \{dX^{-1}\}\,CX + X^{-1}C\,\{dX^{-1}\} \cr &= -\,\{X^{-1}\,dX\,X^{-1}\}\,CX^{-1} - X^{-1}C\,\{X^{-1}\,dX\,X^{-1}\} \cr &= -\,(X^{-1}\,dX\,F + F\,dX\,X^{-1}) \cr &= -\,(X^{-1}\,{\mathbb E}\,F^T + F\,{\mathbb E}\,X^{-T}):dX \cr\cr \frac{\partial F}{\partial X} &= -\,(X^{-1}\,{\mathbb E}\,F^T + F\,{\mathbb E}\,X^{-T}) \cr }$$ where colon denotes the Frobenius Inner Product.
Another common method for dealing with matrix-by-matrix gradients is to use vectorization and the Kronecker Product $$\eqalign{ {\rm vec}(dF) &= -\,{\rm vec}(X^{-1}\,dX\,F + F\,dX\,X^{-1}) \cr df &= -\,(F^T\otimes X^{-1}+X^{-T}\otimes F)\,dx \cr\cr \frac{\partial f}{\partial x} &= -\,(F^T\otimes X^{-1}+X^{-T}\otimes F) \cr }$$