Differentiate matrix quadratic

Question

I wish to differentiate $x^TAx$ wrt. $x_i$ where $x_i$ is the $i$-th element in the vector $x$. I realise when differentiating wrt. $x$ alone the answer is $2Ax$. How would this change when it's $x_i$?

Answer 1

Actually, the vector derivative of $x^TAx$ with respect to $x$ is $(A+A^T)x$. It's only $2Ax$ when $A$ is symmetric.

As for your question, when in doubt, remember you can always expand all vector and matrix notation into components:

$$(Ax)_i = \sum_{j=1}^n A_{ij}x_j$$ $$x^TAx = \sum_{i=1}^n\sum_{j=1}^n x_iA_{ij}x_j.$$

Now if you want to differentiate with respect to $x_k$, you get $$\frac{d}{dx_k}x^TAx = \sum_{j=1}^n A_{kj}x_j + \sum_{i=1}^n x_iA_{ik} = A_{k*}\cdot x + A_{*k}\cdot x,$$ where $A_{k*}$ is the $k$th row of $A$, and $A_{*k}$ is the $k$th column.