Derivative by element from inverse matrices

Question

I have a matrix $\Sigma$ given by \begin{equation*} \Sigma = \begin{bmatrix} \phi_{11} &\phi_{12}\\ \phi_{12} & \phi_{22} \end{bmatrix} \end{equation*}

I'd like to calculate the partial derivative by element, for example, $\frac{\partial y^T\Sigma^{-1} y}{\partial \phi_{11}}$,where $y$ is a vector.

Answer 1

First, computing the derivative of $\Sigma$ is straightforward $$ \frac{\partial\Sigma}{\partial \phi_{11}} = \begin{pmatrix} 1 & 0 \\ 0 & 0 \end{pmatrix}=E_{11} $$ To compute the derivative of $\Sigma^{-1}$ note that $\Sigma^{-1}\Sigma=I$, so by the product rule we have $$ 0 = \frac{\partial I}{\partial \phi_{ij}} = \frac{\partial \Sigma^{-1}\Sigma}{\partial \phi_{ij}}= \frac{\partial \Sigma^{-1}}{\partial \phi_{ij}}\Sigma + \Sigma^{-1}\frac{\partial \Sigma}{\partial \phi_{ij}} $$ Solving for $ \frac{\partial \Sigma^{-1}}{\partial \phi_{ij}}$ in this equation we get the answer that $$ \frac{\partial \Sigma^{-1}}{\partial \phi_{ij}} = -\Sigma^{-1}\frac{\partial \Sigma}{\partial \phi_{ij}}\Sigma^{-1}=-\Sigma^{-1}E_{ij}\Sigma^{-1} $$ If $y$ is independent of $\phi_{ij}$ that means the derivative of your expression would be $$ -y^{\top}\Sigma^{-1}E_{ij}\Sigma^{-1}y $$ Otherwise you would have to use the product rule.

Answer 2

You can compute $\Sigma^{-1}$ as $$\Sigma^{-1}=\frac{1}{\phi_{11}\phi_{22}-\phi_{21}\phi_{12}}\begin{pmatrix}\phi_{22}&-\phi_{12}\\-\phi_{21}&\phi_{11}\end{pmatrix}$$ Then you can multiply this by $y$ and $y^T$ as you described, giving you a scalar.

$$\frac{1}{\phi_{11}\phi_{22}-\phi_{21}\phi_{12}}\left(\phi_{22}y_1^2-\phi_{12}y_1y_2+\phi_{11}y_2^2-\phi_{21}y_1y_2\right)$$

Then differentiate each term.