How to calculate $\frac{\partial \mathrm{logdet}({X}_{i\times i})}{\partial X}$?

Question

I know that $\frac{\partial \mathrm{logdet}(X)}{\partial X}=(X^{-1})^T$.

Lets denote ${X}_{i\times i}$ which is $i\times i$ matrix that is formed from $X\in\mathbb{R}^{n\times n}$ by removing all columns $i+1,i+2,\ldots,n$ and rows $i+1,i+2,\ldots,n$.

Ex: if $X=\begin{bmatrix}1 & 2 & 3\\4 & 5 &6 \\7 & 8 &9\end{bmatrix}$, then $X_{1\times 1}=1$, $X_{2\times 2}=\begin{bmatrix}1 & 2 \\4 & 5 \end{bmatrix}$.

How to calculate $\frac{\partial \mathrm{logdet}({X}_{i\times i})}{\partial X}$?

Answer 1

Just note that using the chain rule you get

$ \left(\frac{\partial\log(\det(X_{i\times i}))}{\partial X}\right)_{lg}= \left(\frac{\partial\log(\det(X_{i\times i}))}{\partial X_{i\times i}}\frac{\partial X_{i\times i}}{\partial X}\right)_{lg}$

It remains to compute $\frac{\partial X_{i\times i}}{\partial X}$ and this is easily done considering that $\left(\frac{\partial X_{i\times i}}{\partial X}\right)_{lj}=\frac{\partial X_{i\times i}}{\partial X_{lj}}=\begin{cases}1\ \ \text{if}\ l,j\le i\\ 0\end{cases}=\begin{pmatrix} &I_{i}&0_{n-i}\\ &0_{n-i}&0_{i}\end{pmatrix}$

Using the result you state at the start of your question we get:

$\frac{\partial\log(\det(X_{i\times I}))}{\partial X}=\begin{pmatrix} &(X_{i\times i}^{-1})^T&0_{n-i}\\ &0_{n-i}&0_{i}\end{pmatrix}$ This is true in general: whenever you consider a function restricted to $X_{i\times i}$, the derivative is going to be: $\frac{\partial f(X_{i\times i})}{\partial X}=\begin{pmatrix} &f'(X_{i\times i})&0_{n-i}\\ &0_{n-i}&0_{i}\end{pmatrix}$

Answer 2

Consider a matrix analog of the first cartesian basis vector. $$\eqalign{ &E &= \pmatrix{I\\{\tt\large 0}} \in {\mathbb R}^{n\times i}\cr }$$ where $I\in{\mathbb R}^{i\times i}$ is the identity matrix and ${\tt\large 0}\in{\mathbb R}^{(n-i)\times i}$ is the zero matrix.

This $E$ matrix can be used to extract the leading $(i\times i)$ block-diagonal of the $X$ matrix, i.e. $$X_{i\times i} = E^TXE \;\doteq\; Y $$ The goal is to calculate the gradient of the function $\,\phi=\log\det(Y)$.
First calculate the differential, then perform a change of variables from $Y\to X$. $$\eqalign{ \phi &= \log\det(Y) \\ d\phi &= (Y^T)^{-1}:dY \\ &= Y^{-T}:E^TdX\,E \\ &= E\,Y^{-T}E^T:dX \\ \frac{\partial \phi}{\partial X} &= E\,Y^{-T}E^T \\ &= \pmatrix{Y^{-T}&0\\0&0} \\ }$$ In some of the steps above, a colon is used to denote the matrix inner product, i.e. $$A:B = {\rm Tr}(A^TB)$$