For the following underdetermined equation system:
$a^Tx=b$ $\qquad$ with: $x = (x_1, x_2, x_3) \in \mathbb{R}^3$, $a=(1, 1/2, 1/4)$ and $b=1$
I want to compute the solution of the following $l_0, l_1$ and $l_2$ regularized problems:
$\min ||x||_0 $ subject to $a^Tx=b$
$\min ||x||_1 $ subject to $a^Tx=b$
$\min ||x||_2 $ subject to $a^Tx=b$
For $l_0$ and $l_1$ regularization the solution seems to be quite obvious:
$x = (1, 0, 0)$
However I'm not quite sure how to calculate the solution for the $l_2$ regularization.
The answer of $l_0$ is all $(x_1,x_2,x_3)$ with either $x_1=0$ or $x_2=0$ or $x_3=0$. For $l_1$ the answer is the only one you obtained and for $l_2$ we need to minimize $x_1^2+x_2^2+x_3^2$ s.t. $x_1+x_2/2+x_3/4=1$. Using Lagrange multipliers we have$$L(x,\lambda)=x_1^2+x_2^2+x_3^2-\lambda(x_1+\dfrac{x_2}{2}+\dfrac{x_3}{4})$$by taking partial differentiation we arrive at:$$2x_1=\lambda\\2x_2=\frac{\lambda}{2}\\2x_3=\frac{\lambda}{4}$$and by substituting in the constraint we have$$\lambda=\frac{32}{21}$$therefore the minimum would be attained when $(x_1,x_2,x_3)=(\dfrac{16}{21},\dfrac{8}{21},\dfrac{4}{21})$ and its value is $$m=(\dfrac{16}{21})^2+(\dfrac{8}{21})^2+(\dfrac{4}{21})^2=\dfrac{256}{21}$$