Gradient of $a'Xa$ not equal to $2Xa$

Question

Let's say

$$a'Xa = \begin{bmatrix} a_1 & a_2 \\ \end{bmatrix} \begin{bmatrix} 1 & 2 \\ 3 & 4 \\ \end{bmatrix} \begin{bmatrix} a_1 \\ a_2 \\ \end{bmatrix}$$

By using common differentiation rules, we have:

$$\frac{\partial a'Xa}{\partial a} = 2Xa= \begin{bmatrix} 2a_1+4a_2 \\ 6a_1 +8a_2 \\ \end{bmatrix}$$

However, as $a'Xa$ can also be expressed as

$$a_1^2+5a_1a_2+4a_2^2$$

when we differentiate it w.r.t. $a$, we'll get

$$\begin{bmatrix} 2a_1+5a_2 \\ 5a_1 +8a_2 \\ \end{bmatrix}$$

Any ideas why these two results are different? Thanks so much!

Answer 1

Your derivative formula is only valid when the matrix is symmetric. Otherwise, it is $$2X_s a=Xa+X^t a$$ Where $X_s$ is the symmetric part of $X$, i.e. $$X_s=\frac{X+X^t}{2}$$ This happens, because in the symmetric+antisymmetric part decomposition of a matrix, we have that $$a^tXa=a^t(X_s+X_a)a=a^tX_sa+a^tX_aa=a^tX_sa$$ Where the antisymmetric part is $$X_a=\frac{X-X^t}{2}$$

Answer 2

You need the matrix $X$ to be symmetric for your result to hold. The general result is $$\frac{\partial}{\partial a} (a' Xa) = \left(X + X'\right)a.$$ See the answers here for example for proofs: Gradient of $x^{T}Ax$.