How to derive the hard thresholding estimator?

Question

The minimization problem is $$ \min_{\mu\in R^p} \sum_{i=1}^p (y_i-\mu_i)^2 + \lambda^2\sum_{i=1}^p \mathbb{1}(\mu_i\neq0) $$ for $y\in R^p$. This is also known as $l_0$ regularization. The solution is given by $$ \hat{\mu_i}= y_i \mathbb{1}(|y_i|>\lambda). $$ Does anybody know how to derive the solution or give some references?

Answer 1

This $p$-dimensional problem decomposes into $p$ one-dimensional problems of minimizing $(y_i-\mu_i)^2+\lambda^2\mathbb 1(\mu_i\ne0)$. You can either choose $\mu_i=0$, or if you don't, the second term is constant and you might as well minimize the first term and choose $\mu_i=y_i$. The solution you give chooses the option of those two that minimizes the sum.