I am trying to find the Jacobian matrix of $x^T A x$. Where $A \in \mathbb R^{n \times n}$ (and $A$ is symmetric) and $x\in \mathbb R^n$.
I have tried to write it in summation notation:
$$x^T A x=\sum_{i=1}^n \sum_{j=1}^n x_ia_{ij}x_j:=y$$
Since $x^TAx=y$ is a polynomial, the corresponding Jacobian matrix will look something like this:
$$J_y=\begin{pmatrix} \frac{\partial f_1}{\partial x_1} & \frac{\partial f_1}{\partial x_2} & ........ & \frac{\partial f_1}{\partial x_n} \end{pmatrix}$$
So the $k$-th component of my Jacobian matrix is:
$$ \frac{\partial y}{\partial x_k}=\frac{\partial }{\partial x_k} \left(\sum_{i=1}^n \sum_{j=1}^n x_ia_{ij}x_j \right) $$
I am stuck here. I have tried separating this into cases where $k=i \not=j$ but I am not really getting anywhere. Is there a way to simplify this further? ( I am not necessarily looking for a complete solution but rather a few hints that will enable me to figure it out by myself. Thanks.)
Define $f:\mathbb R^n \to \mathbb R$ as $f(x)= x^T A x$. Then $f(a+h)-f(a)= 2a^T A h+h^T A h$. As you can see, a candidate for the derivative $\mathrm df(a)$ is $2a^T A$ and one can indeed verify this guess by calculating \begin{equation} \lim _{{h} \to {0}} \frac{\|f({a}+{h}) - f({a}) - 2a^TAh\|}{\|{h}\|}. \end{equation}(Hint: Recall that $h^T A h=\langle h,Ah\rangle$ and use Cauchy Schwarz.)
If you believe $f$ is differentiable, an easier way to find out the derivative of $f$ at $a$ would be to use the result $$\mathrm df(a)y=\left.\frac{\mathrm df}{\mathrm dt}(a+ty)\right|_{t=0}.$$