I have to find the partial derivative of the function $\frac{\partial}{\partial{X}}||X||_1^{2}$
I know that derivative of $||X||_1$ is $-1$ where $||X||_1 > 0$ and $-1$ where $||X||_1$ < 0 and not defined at $||X||_1 = 0$.
Thus using the chain rule can I write the derivative of $||X||_1^{2}$ as:
$-2||X||_1$ where $||X||_1 < 0$ and
$2||X||_1$ where $||X||_1 > 0$
Consider the sign function $${\rm sign}(z) = \begin{cases} +1 &{\rm if\,\,}z\ge 0 \\ -1 &\text{otherwise} \end{cases}$$ applied elementwise to the matrix $\,\,S = {\rm sign}(X)$.
Use this to write the norm. Then find its differential and gradient. $$\eqalign{ \alpha &= \|X\|_1 = S:X \cr d\alpha &= S:dX \cr d(\alpha^2) &= 2\alpha\,d\alpha = 2\alpha\, S:dX \cr \frac{\partial\alpha^2}{\partial X} &= 2\alpha\, S = 2\|X\|_1\,\,{\rm sign}(X) \cr }$$ where a colon denotes the matrix inner product, i.e. $$A:B={\rm Tr}(A^TB)=\sum_{i}\sum_{j}A_{ij}B_{ij}$$
The preceding assumes that the components of $S$ don't change, i.e. $\,dS_{ij}=0$.
In reality, there's a discontinuity at $X_{ij}=0$.