Hessian matrix of $(||x||-b)^2$

Question

I want to calculate the Hessian matrix of $(||x||-b)^2$ wrt $x$, where $||*||$ is the L2 norm and $x$ is a vector. I know how to calculate the first-order derivative: $dV = 2(1 - \frac{b}{||x||})x^Tdx$, but I don't know how to calculte $d^2V$ and get the Hessian matrix. Thanks!

Answer 1

$ \def\c#1{\color{red}{#1}} \def\g#1{\color{blue}{#1}} $Let's use a convention wherein matrices, vectors, and scalars are represented by uppercase latin, lowercase latin, and greek letters, respectively.

Define the variables $$\eqalign{\ \beta &= b \\ \lambda^2 &= \|x\|^2 = x^Tx \quad\implies\quad \lambda\,\c{d\lambda} = x^Tdx \\ }$$ Write the objective function in terms of these variables, then calculate the differential and gradient. $$\eqalign{\ \phi &= (\lambda-\beta)^2 \\ d\phi &= 2(\lambda-\beta)\;\c{d\lambda} \\ &= \frac{2(\lambda-\beta)\;\c{x^Tdx}}{\c{\lambda}} \\ g=\frac{\partial\phi}{\partial x} &= \frac{2(\lambda-\beta)\,x}{\lambda} \\ }$$ Now calculate differential and gradient of $g$, i.e. the Hessian. $$\eqalign{ dg &= \frac{2(\lambda-\beta)\;dx}{\lambda} + \frac{2\;\c{d\lambda}\;x}{\lambda} - \frac{2(\lambda-\beta)x\;\c{d\lambda}}{\lambda^2} \\ &= \frac{2(\lambda-\beta)\,I\,dx}{\lambda} + \frac{2x(\c{x^Tdx})}{\lambda\c{\lambda}} - \frac{2(\lambda-\beta)x(\c{x^Tdx})}{\lambda^2\c{\lambda}} \\ &= \left(\frac{2\lambda^2(\lambda-\beta)I + \g{2\lambda xx^T} - \g{2}{(\g{\lambda}-\beta)\g{xx^T}}}{\lambda^{3}}\right) dx \\ &= \left(\frac{2\lambda^2(\lambda-\beta)I + 2\beta xx^T}{\lambda^3}\right)dx \\ \\ H=\frac{\partial g}{\partial x} &=\left(\frac{2\lambda^2(\lambda-\beta)I+2\beta xx^T}{\lambda^3}\right) \\ &= 2I + \frac{2\beta}{\|x\|}\left(\frac{xx^T}{\|x\|^2}-I\right) \\ }$$