How to find $\frac{\partial x^TAx}{\partial A}$

Question

$y = x^TAx$. I want to find the $\frac{\partial x^TAx}{\partial A}$; that is, $\frac{\partial y}{\partial A}$.

I know the answer is $\frac{\partial y}{\partial A} = xx^T$. I'm not totally clear on how to get that, and I'm really not clear as to why the $x$'s are now $xx^T$. In other words, why is $x^Tx$ not correct?

Note: I take $x$ = to be a column vector.

Answer 1

The differential of your map $f=x^TAx$ is $df=x^TdAx$. It is a linear map sending $H\mapsto x^T H x$, a scalar. Then write $tr(x^T H x)=tr(xx^T H)=\langle\nabla f,H\rangle_F$ with the "gradient vector" $\nabla f=(xx^T)^T=xx^T.$

Answer 2

Considering a more general problem may shed light on the current problem.

Let $$y = x^TAw$$ where $A$ is rectangular, i.e. $\,\,x$ and $w\,$ have different lengths.

Write the function in terms of the Frobenius product and find its differential and derivative as $$\eqalign{ y &= xw^T:A \cr\cr dy &= xw^T:dA \cr\cr \frac{\partial y}{\partial A} &= xw^T \cr }$$ Your problem is a special case of the above, where $\,w=x$.

Answer 3

Let's use Einstein Notation

$y = x_i A_{ij} x_j$

---

$\frac{\partial A_{ij}}{\partial A_{mn}} = \delta_{im}\delta_{nj}$ (4th order tensor)

where $\delta$ is the Kronecker Delta

---

$\frac{\partial y}{\partial A_{mn}} = \frac{\partial (x_i A_{ij} x_j)}{\partial A_{mn}} = x_i \frac{\partial A_{ij}}{\partial A_{mn}}x_j = x_i\delta_{im}\delta_{nj}x_j = x_m x_n$

Back in matricial notation: $\boxed{\frac{\partial y}{\partial A} = x\,x^T}$