Derivative of Schatten $p$-norm

Question

The nuclear norm is a special case of the Schatten $p$-norm. I know how to find out the derivative of nuclear norm, but what is the derivative expression of the matrix Schatten $p$-norm?

Answer 1

For a given matrix $X$, let $$K=\sqrt{X^TX}$$ Then the Schatten norm can be written as $$S_p = \big[{\rm tr}(K^p)\big]^{1/p}$$ and the gradient as $$\eqalign{ \frac{\partial S_p}{\partial X} = \Big(\frac{X}{S_p}\Big)\,\Big(\frac{K}{S_p}\Big)^{p-2}\cr\cr }$$ If the SVD of $X$ is available $$\eqalign{ X &= UDV^T \cr K &= VDV^T \cr S_p &= \big[{\rm tr}(D^{ p})\big]^{1/p} \cr \frac{\partial S_p}{\partial X} &= U\Big(\frac{D}{S_p}\Big)^{p-1}V^T \cr }$$ You can also write simple expressions for the Spectral norm $(p=\infty)$ and its gradient $$\eqalign{ S_\infty &= \sigma_1 \cr \frac{\partial S_\infty}{\partial X} &= u_1\,v^T_1 \cr }$$ where the vectors $\{u_1, v_1\}$ are the first columns of the matrices $\{U,V\}$ and $\{\sigma_1\}$ is the largest singular value, i.e. the $D_{11}$ element of $D$.