Derivative of scalar function with respect to vector

Question

Suppose I have three constant symmetric matrix $\mathbf{M}_{n\times n}$, $\mathbf{C}_{n\times n}$ and $\mathbf{D}_{n\times n}$ and two variable vectors $\mathbf{q}_{n\times 1}$ and $\mathbf{p}_{n\times 1}$. Also I have two function. The first is a matrix that depends only on $\mathbf{q}_{n\times 1}$:

$\mathbf{G}_{n\times n}(\mathbf{q})=\mathbf{M}+\mathbf{C}(\mathbf{q}^T\mathbf{D}\mathbf{q})$

and the second is a scalar function that depends on both $\mathbf{q}_{n\times 1}$ and $\mathbf{p}_{n\times 1}$:

$J(\mathbf{q},\mathbf{p})=\mathbf{p}^T\mathbf{G}^{-1}\mathbf{p}$

I want to find an expression for $\frac{\partial J}{\partial \mathbf{q}}$ in terms of $\mathbf{M}$, $\mathbf{C}$, $\mathbf{D}$, $\mathbf{G}^{-1}$, $\mathbf{q}$ and $\mathbf{p}$.

If I use the formula $d(A^{-1})=-A^{-1}(dA)A^{-1}$, the result will be:

$$\frac{\partial J}{\partial \mathbf{q}}=-\mathbf{p}^T\mathbf{G}^{-1}\frac{\partial \mathbf{G}}{\partial \mathbf{q}}\mathbf{G}^{-1}\mathbf{p}$$

but I can't compute $\frac{\partial \mathbf{G}}{\partial \mathbf{q}}$. Is there another way to solve this problem?

Answer 1

We can salvage your result, if you revert to the differential for a moment: $$ dJ = - p^TG^{-1}dG\,G^{-1}p $$ and expand $dG = C d(q^TDq) = 2C(q^TDdq)$. The term in parentheses is a scalar and doesn't participate in the matrix product, i.e. $$ dJ = -2 p^TG^{-1}C\,G^{-1}p\,\,(q^TDdq) $$ Now we can identify the derivative wrt $q$ as $$ \frac {\partial J} {\partial q} = -2 p^TG^{-1}C\,G^{-1}p\,\,(q^TD) $$

Answer 2

Use the Frobenius (:) Inner Product and rewrite the function as $$\eqalign{ G &= M+C(D:qq^T) \cr J &= pp^T:G^{-1} }$$ In this form, finding the differential and gradient is a bit tedious, but straightforward $$\eqalign{ dJ &= pp^T:dG^{-1} \cr &= -pp^T:G^{-1}\,dG\,G^{-1} \cr &= -G^{-T}pp^TG^{-T}:dG \cr &= -G^{-T}pp^TG^{-T}:(CD:d(qq^T)) \cr &= -\alpha D:d(qq^T) \cr &= -\alpha D:(dq\,q^T + q\,dq^T) \cr &= -\alpha D:2\,{\rm sym}(dq\,q^T) \cr &= -2\alpha\,{\rm sym}(D):dq\,q^T \cr &= -\alpha\,(D+D^T):dq\,q^T \cr &= -\alpha\,(D+D^T)\,q:dq \cr\cr \frac{\partial J}{\partial q} &= -\alpha\,(D+D^T)\,q \cr &= -(G^{-T}pp^TG^{-T}:C)\,(D+D^T)\,q \cr &= -(p^TG^{-1}CG^{-1}p)\,(D+D^T)\,q \cr }$$