Hessian of a function of norm

Question

Does anybody knows how to compute the Hessian of $f(\mathbf{x})=(\|\mathbf{x}\|_2-1)^4$? I cannot figure out how to do that using definition.

Answer 1

It is not pretty. First order of business is computing the first derivative of $f$. This can be done by the chain rule with $g({\bf x}) = \|{\bf x}\|$ and $h(t) = (t-1)^4$. We have that $$Dg({\bf x})({\bf v}) = \frac{\langle {\bf x},{\bf v}\rangle}{\|{\bf x}\|},$$ and so $$\begin{align}Df({\bf x})({\bf v}) &= D(h\circ g)({\bf x})({\bf v}) = Dh(g({\bf x}))\circ Dg({\bf x})({\bf v}) \\ &= Dh(\|{\bf x}\|)\left(\frac{\langle {\bf x},{\bf v}\rangle}{\|{\bf x}\|}\right) = \frac{\langle {\bf x},{\bf v}\rangle}{\|{\bf x}\|} Dh(\|{\bf x}\|)(1) \\ &= \frac{\langle {\bf x},{\bf v}\rangle}{\|{\bf x}\|} h'(\|{\bf x}\|) = \frac{4\langle {\bf x},{\bf v}\rangle}{\|{\bf x}\|}(\|{\bf x}\|-1)^3 \\ &= \left\langle \frac{4(\|{\bf x}\|-1)^3}{\|{\bf x}\|}{\bf x} ,{\bf v}\right\rangle,\end{align}$$which says that $$\nabla f({\bf x}) = \frac{4(\|{\bf x}\|-1)^3}{\|{\bf x}\|}{\bf x}.$$So, the Hessian matrix of $f$ at ${\bf x}$ is the matrix representing the derivative of $\nabla f$ at ${\bf x}$, in the standard basis. So we need to use the chain rule again, and the product rule. Let $\ell(t) = 4(t-1)^3/t$, so that $\nabla f({\bf x}) = \ell(g({\bf x})){\bf x}$, and so: $$\begin{align} {\rm Hess}(f)_{{\bf x}}({\bf v}) =D(\nabla f)({\bf x})({\bf v}) &= D(\ell \circ g)({\bf x})({\bf v}){\bf x} + \ell(g({\bf x})){\bf v} \\ &= D\ell(g({\bf x}))\circ Dg({\bf x})({\bf v}){\bf x} + \frac{4(\|{\bf x}\|-1)^3}{\|{\bf x}\|}{\bf v} \\ &= D\ell(\|{\bf x}\|)\left(\frac{\langle{\bf x},{\bf v}\rangle}{\|{\bf x}\|}\right){\bf x}+ \frac{4(\|{\bf x}\|-1)^3}{\|{\bf x}\|}{\bf v} \\ &= \frac{\langle {\bf x},{\bf v}\rangle}{\|{\bf x}\|}D\ell(\|{\bf x}\|)(1){\bf x}+ \frac{4(\|{\bf x}\|-1)^3}{\|{\bf x}\|}{\bf v} \\ &= \frac{\langle {\bf x},{\bf v}\rangle}{\|{\bf x}\|}\ell'(\|{\bf x}\|){\bf x}+ \frac{4(\|{\bf x}\|-1)^3}{\|{\bf x}\|}{\bf v} \\ &= \frac{\langle {\bf x},{\bf v}\rangle}{\|{\bf x}\|} \frac{12\|{\bf x}\|(\|{\bf x}\|-1)^2 - 4(\|{\bf x}\|-1)^3}{\|{\bf x}\|^2}{\bf x} +\frac{4(\|{\bf x}\|-1)^3}{\|{\bf x}\|}{\bf v} \\ &= \frac{12\langle {\bf x},{\bf v}\rangle(\|{\bf x}\|-1)^2}{\|{\bf x}\|^2}{\bf x} + \frac{4(\|{\bf x}\|^2-1)^3}{\|{\bf x}\|}\left({\bf v} - \langle {\bf x},{\bf v}\rangle\frac{{\bf x}}{\|{\bf x}\|^2}\right). \end{align}$$At least we can see that on the critical points of $f$, where $\|{\bf x}\| = 1$, the Hessian also vanishes. So if you need to find local max/min of $f$, you would have to keep computing higher order derivatives to decide the nature of the critical points. If your goal is to compute the matrix in coordinates, the bilinear form you have to compute is $$({\bf v},{\bf w}) \mapsto \langle {\rm Hess}(f)_{\bf x}({\bf v}), {\bf w}\rangle.$$

---

Edit: just for fun, I did the following computation. If $\varphi:\Bbb R_{\geq 0} \to \Bbb R$ is twice differentiable and $f:\Bbb R^n\to \Bbb R$ is given by $f({\bf x}) =\varphi(\|{\bf x}\|)$, then for every non-zero ${\bf x}$ we have that$$\nabla f({\bf x}) =\varphi'(\|{\bf x}\|)\frac{\bf x}{\|{\bf x}\|}$$and $${\rm Hess}(f)_{\bf x}({\bf v}) = \varphi''(\|{\bf x}\|)\,{\rm proj}_{\bf x}{\bf v} +\frac{\varphi'(\|{\bf x}\|)}{\|{\bf x}\|}\left({\bf v}-{\rm proj}_{\bf x}{\bf v}\right).$$In particular, ${\bf x}$ is a critical point of $f$ if and only if $\|{\bf x}\|$ is a critical point of $\varphi$, in which case $$\langle {\rm Hess}(f)_{\bf x}({\bf v}),{\bf v}\rangle = \varphi''(\|{\bf x}\|)\left(\frac{\langle{\bf x},{\bf v}\rangle}{\|{\bf x}\|}\right)^2.$$

Answer 2

So this is just like taking the second derivative in single variable calculus, except now you have to use the multivariate analogs for the rules of differentiation,

$$f({\bf x}) = g(\|{\bf x}\|),$$

where $g(x)=(x - 1)^4$.

For the gradient you can use the (multivariate) chain rule,

$$\nabla\big(g(\|{\bf x}\|)\big)=\nabla\|{\bf x}\|\cdot g'(\|{\bf x}\|).$$

Now for the Hessian, you can use the multivariate product rule,

$$\text{Hess}_f=\text{Hess}_{\|{\bf x}\|}\cdot g'(\|{\bf x}\|) + \nabla\|{\bf x}\|^T\cdot \nabla\|{\bf x}\|\cdot g''(\|{\bf x}\|).$$

As a sanity check, observe that this expression is symmetric.