Differentiation of $a^\top x x^\top a$

Question

I know that for matrix $A$ and vector $x$, the derivative of the quadratic form with respect to $x$ is

$$\frac{\partial x^TAx}{\partial x} = (A+A^T)x$$

But how do we differentiate $a^T x x^T a$ with respect to $x$?

Answer 1

Hint: Noting that $A^Tx = x^TA \in \Bbb R$, we can rewrite $$ A^TxxA^T = (A^Tx)^2 $$ an appropriate "chain rule" will work here.

Answer 2

I think you mean $a^{T}xx^{T}a=(a^{T}x)(x^{T}a)=(a^{T}x)(a^{T}x)^{T}={(a^{T}x)}^{2}.$ You're placement of the transpose operator needs adjustment. This is because $a,x\in\mathbb{R}^{n\times1}$ implies $a^{T}x\in\mathbb {R}$. Also, using the uppercase $A$ makes it seem like $A\in\mathbb{R}^{n\times m}$ in which case $A^{T}x\in\mathbb{R}^{m\times 1}$ and ${(A^{T}x)}^2$ is only well-defined if and only if $m=1$.

Answer 3

Since $\mathrm a^\top \mathrm x$ is a scalar and scalar multiplication is commutative,

$$\nabla_{\mathrm x} \left( \mathrm a^\top \mathrm x \mathrm x^\top \mathrm a \right) = \nabla_{\mathrm x} \left( \mathrm x^\top \mathrm a \mathrm a^\top \mathrm x \right) = \color{blue}{2 \,\mathrm a \mathrm a^\top \mathrm x}$$

because rank-$1$ matrix $\mathrm a \mathrm a^\top$ is symmetric.

---

matrix-calculus scalar-fields gradient