Finding the Hessian of matrices?

Question

The function is given in matrix form as $f(x)=x^TAx $

How would you find the Hessian of such a function?

Answer 1

We have that

$$\frac {\partial^2 (x^TAx)}{\partial x_i^2}=\frac {\partial^2 (a_{ii}x_i^2)}{\partial x_i^2}=2a_{ii}$$

$$\frac {\partial^2(x^TAx)}{\partial x_ix_j}=\frac {\partial^2 (a_{ij}x_ix_j+ a_{ji}x_jx_i)}{\partial x_ix_j}=a_{ij}+a_{ji}$$

therefore the Hessian is given by $A+A^T$.

Answer 2

$\mathbf{x}^{\top}\mathbf{A}\mathbf{x} = \sum_{i,j =1}^{n} a_{i,j} x_i x_j$

$[\mathbf{H}]_{ij} = \frac{\partial^{2} \sum_{i,j =1}^{n} a_{i,j} x_i x_j}{\partial x_i \partial x_j}= a_{i,j} +a_{j,i}$

$\mathbf{H} = \mathbf{A} +\mathbf{A}^{\top}$

Answer 3

If $x=(x_1,\cdots,x_k)$, the $(i,j)$ element of $A$ is $a_{ij}, 1\le i,j\le k$, then $$x^TAx=\sum_{i=1,j=1}^{k,k} a_{ij}x_ix_j$$ so the $(i,j)$ element of the Hessian of $x^TAx$ will be $$\dfrac{\partial^2 (x^TAx)}{\partial x_i\partial x_j}=\dfrac{\partial}{\partial x_i}\dfrac{\partial (x^TAx)}{\partial x_j}\\=\dfrac{\partial}{\partial x_i}\left(\sum_{p=1}^k a_{pj}x_p+\sum_{q=1}^k a_{jq}x_q\right)=a_{ij}+a_{ji}$$ which gives the $(i,j)$ element of $A+A^T$, so we must have the Hessian $H=A+A^T$.