What is the gradient of $x^T A\, x$ with respect to the matrix $A$?

Question

I have seen many times that the gradient of $x^TA\,x$ with respect to $x$ is $2A\,x$. But how do you find its gradient with respect to $A$?

Answer 1

You can just write $$ f(A)=x^TA\,x= \sum_{i,j}a_{ij}x_i x_j $$ Therefore $${\partial f \over \partial a_{ij}}=x_i x_j$$ and the gradient is the matrix with entries $(\nabla f)_{ij}=x_i x_j$, that is $$ \nabla f = x\,x^T. $$