Second derivative of a matrix quartic form

Question

I need to compute the second derivative of the following quartic expression: $$x^H A^H x x^H A x$$ where is Hermitian. I have tried to compute the first derivative, and if I am not wrong, it should be: $$(A+A^H) x x^H (A+A^H) x$$ But then, I do not know how to proceed to calculate the second derivative. Could someone sketch the steps I need to follow? Thank you.

Answer 1

Define the scalar variables $$\eqalign{ &\phi &= x^HAx = (A^Tx^*)^Tx \cr &\phi^* &= x^HA^Hx = (A^*x^*)^Tx \cr &\psi &= \phi^*\phi \cr }$$ Find the gradient of your function $(\psi)$ with respect to $x$, treating $x^*$ as an independent variable. $$\eqalign{ d\phi &= (A^Tx^*)^T\,dx \cr d\phi^* &= (A^*x^*)^T\,dx \cr d\psi &= \phi\,d\phi^* + \phi^*\,d\phi \cr &= (\phi A^*x^* + \phi^*A^Tx^*)^Tdx \cr g = \frac{\partial\psi}{\partial x} &= A^*x^*\phi + A^Tx^*\phi^* \cr g^* = \frac{\partial\psi}{\partial x^*} &= Ax\phi^* + A^Hx\phi \cr }$$ Not that we need it, but the last equation is a consequence of the fact that $\psi=\psi^*\,$ (it's real).

Now the Hessian is just the gradient of the gradient, so $$\eqalign{ dg &= (A^*x^*)\,d\phi + (A^Tx^*)\,d\phi^* \cr &= \Big((A^*x^*)(A^Tx^*)^T + (A^Tx^*)(A^*x^*)^T\Big)\,dx \cr H = \frac{\partial g}{\partial x} &= A^*x^*x^HA + A^Tx^*x^HA^H \cr }$$ Note that the Hessian is symmetric, but it is not Hermitian.

Answer 2

The second derivative of a multi-variable function $f=f(x_1,\ldots,x_n)$ is usually expressed as its Hessian matrix $$ Hf(x_1,\ldots,x_n)=\left(\frac{\partial^2 f}{\partial x_i\partial x_j}\right)_{i,j=1}^n $$ Here $$ f(x)=x^tA^txx^tAx=(x^tBx)^2, \quad \text{where $B=\frac{1}{2}(A+A^t)=(b_{ii})$}. $$ Hence $$ \frac{\partial f}{\partial x_i}=4(x^tBx)(Bx)_i $$ and $$ \frac{\partial^2 f}{\partial x_i\partial x_j}=8(Bx)_i(Bx)_j+4(x^tBx)b_{ij}, $$ and thus $$ Hf(x)=8Bx(Bx)^t+4(x^tBx)B $$