The $\ell_p$-norm of a vector $\mathbf{x}$ is defined as the root of the sum of the absolute values of its elements raised to the power of $p$.
I want to calculate the following derivative. $$\frac{\partial ^2}{\partial \mathbf{x}^2}\Vert \mathbf{A}^{1/p}\mathbf{x} \Vert_p ^p.$$
I think the first derivative is (not sure if it is correct):
$$\frac{\partial }{\partial \mathbf{x}}\Vert \mathbf{A}^{1/p}\mathbf{x} \Vert_p ^p = p \mathbf{A}^{1/p} |\mathbf{A} ^{1/p} \mathbf{x} |^{p−1},$$
where $| \cdot |$ is the absolute value. Struggling to compute the second derivative.
Help would be appreciated.
$ \def\l{\lambda} \def\o{{\tt1}} \def\ll{\ell_p} \def\LR#1{\left(#1\right)} \def\op#1{\operatorname{#1}} \def\abs#1{\op{abs}\LR{#1}} \def\Diag#1{\op{Diag}\LR{#1}} \def\sign#1{\op{sign}\LR{#1}} \def\trace#1{\op{Tr}\LR{#1}} \def\frob#1{\left\| #1 \right\|_F} \def\qiq{\quad\implies\quad} \def\p{\partial} \def\grad#1#2{\frac{\p #1}{\p #2}} \def\hess#1#2#3{\frac{\p^2 #1}{\p #2\,\p #3}} \def\c#1{\color{red}{#1}} \def\CLR#1{\c{\LR{#1}}} $For typing convenience, define the variables $$\eqalign{ \o &= \{{\rm vector}\,&{\rm of\:ones}\} \\ M &= A^{1/p} \\ v &= Mx &\qiq \c{dv = M\:dx} \\ b &= \abs{v} &\qiq b\odot b = v\odot v \\ B &= \Diag{b} &\qiq Bv = b\odot v \\ V &= \Diag{v} &\qiq V\o = v\odot\o = v \\ s &= \sign{v} &\qiq v = s\odot b,\; s = v\oslash b \\ \l &= \LR{\ll}^p \\ }$$ and the Frobenius product, which is a concise notation for the trace $$\eqalign{ A:B &= \sum_{i=1}^m\sum_{j=1}^n A_{ij}B_{ij} \;=\; \trace{A^TB} \\ A:A &= \frob{A}^2 \\ }$$ The following relationships will prove useful $$\eqalign{ v\odot b &= b\odot v &\qiq Vb = Bv = BV\o \\ b\odot b &= v\odot v &\qiq b\odot db = v\odot dv \\ B^2 &= V^2 &\qiq \c{B\:db = V\,dv} \\ VB &= BV &\quad\{ {\rm diagonal\;matrices\;commute} \} \\ }$$ Use the above notation to write the objective function and calculate its gradient $$\eqalign{ \l &= \o:b^{\odot p} \\ d\l &= \o:\LR{p\,b^{\odot p-1}\odot db} \\ &= p\,b^{\odot p-2}:\LR{b\odot db} \\ &= p\,b^{\odot p-2}:\LR{v\odot dv} \\ &= p\LR{b^{\odot p-2}\odot v}:dv \\ &= p\LR{B^{\,p-2}\,v}:\CLR{M\,dx} \\ &= p\LR{M^TB^{\,p-2}\,v}:dx \\ \grad{\l}{x} &= pM^TB^{\,p-2}\,v \;=\; g \quad \{ {\rm gradient} \} \\ }$$ This is almost the same as your result, but you missed the transpose on the $M$ matrix.
Now calculate the gradient of $g$ aka the Hessian $$\eqalign{ dg &= pM^TB^{\,p-2}\,dv + pM^TdB^{\,p-2}\,v \\ &= pM^TB^{\,p-2}\,dv + p(p-2)M^TB^{\,p-3}\,dB\;v \\ &= pM^TB^{\,p-2}\,dv + p(p-2)M^TVB^{\,p-4}\CLR{B\;db} \\ &= pM^TB^{\,p-2}\,dv + p(p-2)M^TVB^{\,p-4}\CLR{V\,dv} \\ &= pM^TB^{\,p-2}\,dv + p(p-2)M^TB^{\,p-2}\,dv \\ &= \LR{p^2-p}M^TB^{\,p-2}\,\CLR{M\,dx} \\ \grad{g}{x} &= \LR{p^2-p}M^TB^{\,p-2}\,M \;=\; H \quad \{ {\rm Hessian} \} \\ }$$ As a sanity check, note that $H$ is a symmetric matrix.