Let $x \in \mathbb{R}^{n}$ a vector, and $A(x) \in \mathbb{R}^{n \times n}$ a symmetric matrix depending on $x$ I need to find a way to differentiate $p(x) = x^t A(x) x \in $ with respect to $x$. This should be a 1 by $n$ vector.
I tried the following \begin{align} \frac{d p(x)}{d x} & = \frac{d x^t A(x) x }{d x} \\ & = \frac{d x^t}{d x} A(x) x + x^t \frac{d A(x) x }{d x} \end{align} But I think this is already incorrect because the first term can't be an 1 by $n$ vector anymore. Does anyone know how to do this?
The derivative of such functions from $\Bbb R^n$ to $\Bbb R$ is usually interpreted as being the gradient $\mathrm{grad}\; p$. When rewriting $p$ in components
$$x^\top A(x)x=\sum_{i,j}x_ix_jA_{ij}(x)$$
you can apply the gradients definition immediately. Let $\partial_k$ be the partial derivative w.r.t. the $k$-th component of $x$, then
\begin{align} \partial_k p(x) &=\sum_j(x_jA_{kj}(x)+x_kx_j[\partial_kA_{kj}(x)])+\sum_{i\not=k,j}x_ix_j[\partial_kA_{ij}(x)]\\ &=\sum_jx_jA_{kj}(x)+\sum_{i,j}x_ix_j[\partial_kA_{ij}(x)]\\ &=e_k^\top A(x)x+x^\top [\partial_kA(x)] x. \end{align}
Take these together to a vector to obtain
$$\text{grad }p(x) = A(x)x+(\partial_1A(x)x \;\cdots\; \partial_nA(x)x)^\top x,$$
where in the second term there is this matrix with rows $\partial_kA(x)x$ (after transposing).