sparse norm for optimization problem

101 Views Asked by At

I want to solve an optimization problem in general form:

$$\arg \min f(x) + \lambda *g(x)$$

and i want to choose / define a $g(x)$ in a way to have a sparse solution such that between two possible answers $A=\begin{bmatrix}1\\0\\0\end{bmatrix}$ and $B=\begin{bmatrix}0.5\\0.5\\0\end{bmatrix}$, it chooses A.

And of course $\|A\|_1=\|B\|_1$ and $\|A\|_2 > \|B\|_2$.

I know that using $\|A\|_p$ with $p<1$ could be a good criteria, but it is NP-hard problem and makes the optimization problem more difficult.

I also noticed $sign(A)$ is another good guess, but then it is not diffrentiable in from the optimization point of view!

1

There are 1 best solutions below

3
On

As someone suggested earlier that for getting sparse solution in polytime, you should use $\|\cdot\|_1$ norm. Very general and fast methods to solve minimization problem of type $f(x)+\lambda g(x)$ are known as proximal algorithms (check wiki or paper). Here, $f(\cdot)$ is data fitting term which is convex and differentiable whereas $g(\cdot)$ is any regularizer (which in your case is 1-norm).