Differentiation of $x^TAx$

Question

I have in my text that if I differentiate $x^TAx$ with respect to the vector $x$ I get $2xA$ - could I ask why?

Here $x$ is a $3\times1$ vector, $A$ is a $3\times 3$ matrix - I am given the explanation that $x^2A$ behaves like $x^TAx$.

Answer 1

The derivation with sums isn't all that bad. We find that $$ \frac{\partial }{\partial x_i} x^TAx = \frac{\partial }{\partial x_i} \sum_{j=1}^n \sum_{k=1}^n a_{jk}x_i x_j = \\ \sum_{k \neq i} a_{ik} x_k + \sum_{j \neq i} a_{ji} x_j + 2 a_{ii} x_i = \\ \sum_{k=1}^n a_{ik} x_k + \sum_{j=1}^n a_{ji} x_j = \\ [Ax]_i + [A^Tx]_i $$ Alternatively, it suffices to note that $$ (x+h)^TA(x+h) - x^TAx = h^TAx + x^TA h + O(\|h\|^2) $$ So that $$ Df_x(h) = h^TAx + x^TAh = x^TA^Th + x^TAh = x^T(A^T + A)h $$ Which is to say that $Df_x = x^T(A + A^T)$.

Answer 2

If $A$ is a self-adjoint operator $A : \mathbb R^m \to \mathbb R$ then one may take the quadratic form $\varphi : \mathbb R^n \to \mathbb R$ given by $\varphi (x) = \langle Ax , x\rangle$ and its derivative (of a bilinear form) is given by

$$\varphi'(x) \cdot v = \langle Av, x\rangle + \langle Ax,v\rangle = 2\langle Ax, v\rangle$$

since $\langle Av, x\rangle = \langle v,Ax\rangle$.

Now one may prefer to consider $A$ as the symmetric $m \times m$ matrix $\textbf a$ and $x \in \mathbb R^m$ as a $m \times 1$ matrix $\textbf x$ whose transpose is the $1 \times m$ matrix $\textbf {x}^T$, then the quadratic form $\varphi$ is written as $\varphi (\bf x) = \bf {x}^T \bf a \bf x$. From this

$$\varphi'(\bf x) \cdot v = \bf v^T \bf a \bf x + x^Tav = x^T a v + x^Ta v = 2x^Ta v$$