Derivative/Gradient of log $l_1$ norm

560 Views Asked by At

As derivative of $l_{p}$- norm is \begin{align*} \frac{\partial}{\partial\mathbf{x}}{||\mathbf{x}||}_{p} &= \frac{\mathbf{x} |\mathbf{x}|^{p-2}}{{||\mathbf{x}||}_{p}^{p-1}} \end{align*}

I want to find $\nabla log(||H||_{1})$, where $H$ is positive matrix. So, the chain rule is \begin{align*} \nabla log(||H||_{1}) &= log(||H||_{1})' (||H||_{1})'\\ &= \frac{1}{||H||_{1}}\frac{H |H|^{1-2}}{{||H||}_{1}^{1-1}}\\ &= \frac{1}{||H||_{1}}\\ \end{align*}

But the answer seems to be

\begin{align*} \nabla log(||H||_{1}) &= &= \frac{H}{||H||_{1}}\\ \end{align*}

What am I missing here? Where is $H$ at numerator from?

1

There are 1 best solutions below

2
On BEST ANSWER

Apply the sign function element-wise to the matrix $H$ $$S = {\rm sign}(H)$$ Use this to write the Manhattan norm as $$\eqalign{ \|H\|_1 &= S:H \cr }$$ where the colon denotes the trace/Frobenius product, i.e. $\,\,A:B={\rm tr}(A^TB).$

Use this to calculate the logarithmic derivative of the Manhattan norm as
$$\eqalign{ \Omega &= \log(\|H\|_1) \cr d\Omega &= \frac{d\|H\|_1}{\|H\|_1} = \frac{S:dH}{\|H\|_1} \cr \frac{\partial\Omega}{\partial H} &= \frac{S}{\|H\|_1} \cr\cr }$$ If all elements of $H>0\,$ then the numerator is the matrix $S=1$ (as you expected).
In any case, the numerator is definitely not $H$.