Derivative of Inverse of sum of matrices

Question

Given is the function $f : \mathbb{R}^p \to \mathbb{R}$ with

$$ f(x) = q(x)^{\top} G^{-1} q(x) $$

where $G = A + x_1 B_1 + \ldots + x_p B_p$.

• The matrices $A, B_1, \ldots, B_p \in \mathbb{R}^{n \times n}$ are all symmetric positive definit.

• $q: \mathbb{R}^n \to \mathbb{R}^n$ and the Jacobian $\nabla q$ is known.

Is it possible to derive a closed form for $\nabla f$? For me, the hard part is $G^{-1}$. Any hints or suggestions are really appreciated!

Answer 1

$$\frac1{\Delta x}\big[(A+(x+\Delta x)B)^{-1}-(A+xB)^{-1}\big]=$$ $$\frac1{\Delta x}(A+xB)^{-1}(A+xB)\big[(A+(x+\Delta x)B)^{-1}-(A+xB)^{-1}\big] (A+(x+\Delta x)B) (A+(x+\Delta x)B)^{-1}$$ $$=-(A+xB)^{-1}B (A+(x+\Delta x)B)^{-1}\rightarrow-(A+xB)^{-1}B(A+xB)^{-1}$$ as $\Delta x\rightarrow0$.

So that $\partial f/\partial x_j=2\partial q/\partial x_j^\top G^{-1} q -q^\top G^{-1}B_jG^{-1}q$.

Answer 2

$ \def\l{\left} \def\r{\right} \def\o{{\tt1}} \def\p{{\partial}} \def\g#1#2{\frac{\p #1}{\p #2}} \def\c#1{\color{red}{#1}} \def\B{{\mathbb B}} \def\R{{\mathbb R}^p} \def\E{{\cal E}} $Let $\{e_k\}$ denote the standard basis vectors for $\R$ and $M$ the gradient of $q,\;$ then $$M = \g{q}{x} \quad\implies\quad dq = M\,dx$$ The gradient of a matrix $(G)$ with respect to a vector $(x)$ is a third-order tensor $(\B)$ with components $$\eqalign{ \B_{ijk} &= \g{G_{ij}}{x_k} \\ }$$ Using this tensor we can write the other terms in the problem as $$\eqalign{ B_k &= \B\cdot e_k \;\;\implies\;\; \B = \sum_{k=1}^p B_k\star e_k \\ \B\cdot x &= \sum_{k=1}^p B_k\star\l(e_k\cdot x\r) = \sum_{k=1}^p B_k x_k \\ G &= A + \B\cdot x \\ }$$ where $(\star)$ denotes the tensor/dyadic product, and $(\cdot)$ the dot standard product.

Let's also define a new vector $(w)$ with components $$\eqalign{ w_k = \sum_{k=1}^p G^{-1}qq^TG^{-1}:B_k \\ }$$ where $(:)$ denotes the double-dot product and is an alternative notation for the trace, i.e. $$\eqalign{ A:B &= \sum_{i=1}^m\sum_{j=1}^n A_{ij}B_{ij} \;=\; {\rm Tr}\!\l(AB^T\r) \\ A:A &= \big\|A\big\|_F^2 \\ }$$ Now we can write the function and calculate its differential and gradient. $$\eqalign{ f &= q^TG^{-1}q \\ &= G^{-1}:qq^T \\ df&= G^{-1}:\l(dq\,q^T+q\,dq^T\r) + qq^T:dG^{-1} \\ &= \l(G^{-1}+G^{-T}\r):dq\,q^T - qq^T:G^{-1}dG\,G^{-1} \\ &= 2G^{-1}:dq\,q^T - G^{-1}qq^TG^{-1}:dG \\ &= 2G^{-1}q:M\,dx - G^{-1}qq^TG^{-1}:\B\cdot dx \\ &= \l(2MG^{-1}q - G^{-1}qq^TG^{-1}:\B\r)\cdot dx \\ &= \l(2MG^{-1}q - w\r)\cdot dx \\ \g{f}{x} &= 2MG^{-1}q - w \\ }$$