What do you call an L1 regularization term involving a matrix vector product

100 Views Asked by At

Often times I have difficulties finding certain things when I do not use the correct terminology. Or there are many different terms for the same thing. I would like to know what you would call the following optimization problem, so I have an idea what to look for:

$ \begin{align} \min\limits_{x} ||Ax-b||^2_2 + \lambda \mathcal{R}(x) \end{align} $

where the regularizer is of the form

$ \mathcal{R}(x) = ||Dx||_1 $

What if $D$ is just a positive diagonal matrix? Would it be called differently?

Thank you very much for your answers.

1

There are 1 best solutions below

1
On BEST ANSWER

If by positive you mean strictly positive, so that all the diagonal elements are $>0$, then $D$ is invertible and your problem is equivalent to $$\min_{y}\|By-b\|^2_2+\lambda\|y\|_1\,,$$ where $B = AD^{-1}$ and you're using the standard $L^1$-regularisation.

If you're interested in the $x^\star$ achieving the minimum in your original problem, it is simply given by $$x^\star = D^{-1}y^\star\,,$$ where $y^\star$ achieves the minimum in the standard-regularised problem.

EDIT

Note that $D$ does not even need to be diagonal or positive defined, just invertible, in order to go back to the original $L^1$-regularisation.