Gradient of a matrix expression

Question

What would be $\nabla f(x)$ if $f(x) = x^tAx$ where A is a n by n matrix and x is a n by 1 matrix. I have gone till $x'Ax + x^tAx'$ (where x' represents derivative) to find the solution but can see how I can write this as $(A^t + A)x$?

Answer 1

Before we start deriving the gradient, some facts and notations for brevity:

• Trace and Frobenius product relation $$\left\langle A, B C\right\rangle={\rm tr}(A^TBC) := A : B C$$
• Cyclic properties of Trace/Frobenius product \begin{align} A : B C &= BC : A \\ &= A C^T : B \\ &= {\text{etc.}} \cr \end{align}

Let $f(x) := x^T A x = x:Ax$.

Now, we can obtain the differential first, and then the gradient. \begin{align} df &= d\left( x:Ax \right) \\ &= \left(dx : Ax\right) + \left(x : A \ dx\right) \\ &= \left(Ax : dx\right) + \left(A^Tx : dx\right) \\ &= \left( A + A^T\right)x : dx\\ \end{align}

Thus, the gradient is \begin{align} \frac{\partial}{\partial x}f(x) = \frac{\partial}{\partial x} \left\{x^T A x\right\} = \left( A + A^T\right)x. \end{align}