Hessian of a composition of maps

Question

I have three maps: $$f:\mathbb{R}^m\rightarrow\mathbb{R}^n\,,\quad g:\mathbb{R}^n\rightarrow\mathbb{R}^p\,,\quad h:\mathbb{R}^p\rightarrow\mathbb{R}$$ and I would like to compute the Hessian matrix $H_{h\circ g\circ f}$ of the composite map $h\circ g\circ f$.

For some reasons I would like to perform the computation in this way: $$H=\frac{\partial}{\partial x}\left(J_{h\circ g}\right)\cdot J_f$$ where $J$ is the Jacobian matrix of the map subscripted.

Applying the product rule I should obtain: $$J_f^t \cdot H_{h\circ g}\cdot J_f + (\text{another term}).$$

Now I am wondering why I obtain the first term. In particular, why do I have to left-multiply for the transpose of the Jacobian (except for the fact that I need to obtain a symmetric matrix)?

Answer 1

Think about the one dimensional case with two functions $f$ and $g$ and composition $h=g\circ f$.

We have $$h'(x)=g'(f(x))f'(x)$$

so that

$$h''(x)=g''(f(x))[f'(x)]^2+g'(f(x))f''(x)\qquad \qquad \quad\ $$

$$=f'(x) \cdot g''(f(x))\cdot f'(x)+g'(f(x))f''(x)$$

Answer 2

In matrix form, you also have

$$\nabla^2 (f \circ g)(x) = J^\top_g H_f J_g + \sum_{i=1}^n \frac{\partial f}{\partial y^i} \cdot H_g^i$$

This formula holds for $f : \mathbb{R}^n \to \mathbb{R}$ and $g : \mathbb{R}^m \to \mathbb{R}^n$