I need to find the second order derivative of a matrix.
$f(\pmb{\Delta})=\log \det(\pmb{I}+k\pmb{V^T \Delta V})$
Where $ \pmb{\Delta}$ is a triangular matrix.
I did the first order derivative but not sure if is correct. $$\frac{\partial f(\pmb{\Delta})}{\partial [\pmb{\Delta}]_{ii}} = \left[ k\pmb{V} \left( \pmb{I}+k\pmb{V^H}\pmb{\Delta}\pmb{V} \right)^{-1}\pmb{V^H} \right]_{ii} $$
This is correct? How about the second order derivative?
Define the matrix variables $$\eqalign{ X &= \Delta \cr A &= I + kV^TXV \implies dA &= kV^T\,dX\,V \cr Y &= kVA^{-T}V^T \cr }$$ Write the function in terms of these variables and calculate its differential and gradient. $$\eqalign{ f &= \log\det A \cr df &= A^{-T}:dA \cr &= A^{-T}:kV^T\,dX\,V \cr &= kVA^{-T}V^T:dX \cr G = \frac{\partial f}{\partial X} &= kVA^{-T}V^T \cr }$$ Repeat the process for the gradient. $$\eqalign{ dG &= kV\,dA^{-T}\,V^T \cr &= -kVA^{-T}\,dA^T\,A^{-T}V^T \cr &= -k^2VA^{-T}V^T\,dX^T\,VA^{-T}V^T \cr &= -Y\,dX^T\,Y \cr &= -Y{\mathcal E}Y:{\mathcal L}:dX \cr {\mathcal H} = \frac{\partial G}{\partial X} &= -Y{\mathcal E}Y:{\mathcal L} \cr }$$ where $({\mathcal E},{\mathcal L})$ are 4th order tensors whose components can be written in terms of Kronecker deltas $$\eqalign{ {\mathcal E}_{ijkl} &= {\delta}_{ik} \, {\delta}_{jl} \cr {\mathcal L}_{ijkl} &= {\delta}_{il} \, {\delta}_{jk} \cr }$$ Note that the second derivative (aka Hessian) is itself a 4th order tensor.
Double and single contraction products were used in some of the steps above, which can be expressed in component form as $$\eqalign{ {\mathcal B}_{ikln} &= \big(-Y{\mathcal E}Y\big)_{ikln} &= \sum_j\sum_m -Y_{ij}{\mathcal E}_{jklm}Y_{mn} \cr {\mathcal H}_{ijmn} &= \big({\mathcal B}:{\mathcal L}\big)_{ijmn} &= \sum_k\sum_l {\mathcal B}_{ijkl}{\mathcal L}_{klmn} \cr A:X &= \sum_i\sum_j A_{ij}X_{ij} &= {\,\rm Tr}\big(A^TX\big) \cr }$$