What is the derivative of $x^TAx$ respect to matrix $A$ where $x$ is a vector? Isn't it $xx$?
Below is what I calculate, please help me to check where the problem is:
$\frac{d}{dA}x^TAx=\frac{dx^TA}{dA}x+x^TA\frac{dx}{dA}=\frac{dx^TA}{dA}x=xx$.
Thanks very much.
$$\frac{\partial}{\partial a_{i,j}} (x^\top A x) = \frac{\partial}{\partial a_{i,j}} \sum_{i'=1}^n \sum_{j'=1}^n a_{i', j'}x_{i'} x_{j'} = x_i x_j,$$ so if you put all the partial derivatives into a matrix whose $i,j$ entry is $x_ix_j$, you obtain $xx^\top$.