How do I take the gradient of a quadratic form with two matrix variables?

Question

Usually quadratic form gradients are like $\nabla f(x)$ where $f(x) = x^TAx$.

However, I need to take the gradient with a second variable (and that variable is a matrix): $\nabla f(x, Y)$ where $f(x,Y) = x^TY^{-1}x$. How do I find the gradient here?

Answer 1

The inner/Frobenius product is a convenient product notation for the trace operation $$A:B={\rm tr}(A^TB)$$ Use the Frobenius product to rewrite the function. In this form it's easy to calculate the differential, and then the gradients. $$\eqalign{ f &= Y^{-1}:xx^T \cr \cr df &= Y^{-1}:d(xx^T) + xx^T:dY^{-1} \cr &= Y^{-1}:(dx\,x^T+x\,dx^T) - xx^T:Y^{-1}\,dY\,Y^{-1} \cr &= (Y^{-1}+Y^{-T})x:dx - Y^{-T}xx^TY^{-T}:dY \cr &= 2Y^{-1}x:dx - Y^{-1}xx^TY^{-1}:dY \cr \cr \frac{\partial f}{\partial x} &= 2Y^{-1}x,\,\,\,\,\,\,\,\, \frac{\partial f}{\partial Y} = -Y^{-1}xx^TY^{-1} \cr \cr }$$ In the last few steps, I made use of the symmetry of $Y$.