Derivative of matrix term wrt to matrix

Question

Assume we got matrices $X_{n\times n}$, $A_{n\times n}$ and $\Delta_{n\times n}$. The relation between these are as $X=A+\Delta$. Then how can one calculat the following derivative? Are there any assumptions for it? $$\frac{d(I-A)^{-1}}{dX}=\frac{d(I-X+\Delta)^{-1}}{dX}\,.$$

Answer 1

Consider $$(I-A)^{-1}(I-A)=I$$ So, by the Product Rule we have $$\frac{d(I-A)^{-1}}{dX}(I-A)+(I-A)^{-1}\frac{d(I-A)}{dX}=0$$ that is $$\frac{d(I-A)^{-1}}{dX}=-(I-A)^{-1}\frac{d(I-A)}{dX}(I-A)^{-1},$$ using (and assuming that delta does not depends on $X$) $$\frac{d(I-X+\Delta)}{dX}=-I$$ we got $$\frac{d(I-A)^{-1}}{dX}=-(I-A)^{-1}(-I)(I-A)^{-1}.$$

Answer 2

$ \def\e{{\varepsilon}} \def\B{B^{-1}} \def\Bij{B_{ij}} \def\Xkl{X_{kl}} \def\LR#1{\left(#1\right)} \def\op#1{\operatorname{#1}} \def\trace#1{\op{Tr}\LR{#1}} \def\qiq{\quad\implies\quad} \def\p{\partial} \def\grad#1#2{\frac{\p #1}{\p #2}} \def\c#1{\color{red}{#1}} \def\CLR#1{\c{\LR{#1}}} $Let $\,\e_k\in{\mathbb R}^n$ denote the standard basis vectors and note that $$\eqalign{ X &= \Xkl\LR{\e_k\e_l^T} \qiq \grad{X}{\Xkl} = \e_k\e_l^T \\ }$$ For typing convenience, define the matrix variables $$\eqalign{ A &= X-\Delta &\qiq dA = dX \\ B &= \LR{I-A}^{-1} &\qiq d\B = -dA \\ }$$ Now differentiate the $B$ matrix, starting with the definition of the matrix inverse $$\eqalign{ I &= B\B \\ 0 &= dB\;\B + B\;d\B \\ dB &= -B\;d\B\;B \\ &= +B\;dA\;B \\ &= B\;dX\;B \\ \grad{B}{\Xkl} &= B\LR{\e_k\e_l^T}B \\ \grad{\Bij}{\Xkl} &= \e_i^T\LR{B\,\e_k\,\e_l^TB}\e_j \;=\; B_{ik}\,B_{lj} \\ }$$ Notice that the resulting gradient expression has four free indices, therefore it represents a 4th order tensor, not a matrix.

However, the component-wise gradient in the preceding line is a matrix.