Given a vector $y$ and a matrix $X$ (not necessarily square) we consider the $l^1$-penalized least squares problem of minimizing
$$f(b) :=\frac{1}{2}\Vert y-Xb \Vert_2^2+\lambda \Vert b \Vert_1 $$
over all $b$. It is easy to see that $b=0$ if we choose $\lambda>0$ large enough.
But is it possible to find an explicit bound $C$ (depending on $y$ and $X$) such that $\hat b = \operatorname{argmin}_b f(b) = 0$ for all $\lambda \geq C$?
My Idea so far: When we consider the 1D case we just have $f(b) = \underbrace{ \frac{1}{2}(y-xb)^2 }_{=:g(b)}+ \lambda |b|$ and we see that the $|b|$ term dominates as long as $|g'(\hat b)| < \lambda$, i.e. $f'(\hat b - \epsilon) < 0$ and $f'(\hat b + \epsilon) > 0$. But is it possible to generalize this result?
The vector $b=0$ being a minimizer means that $$ \frac12 \|y\|_2^2 \le \frac{1}{2}\| y-Xb \|_2^2+\lambda \|b\|_1 \tag1$$ for all $b$. Expand $\| y-Xb \|_2^2$ as $\|y\|^2 - 2\langle y, Xb\rangle +\|Xb\|_2^2$ and rearrange (1) as $$ \langle y, Xb\rangle \le \lambda \|b\|_1 + \frac{1}{2}\|Xb\|_2^2 \tag2$$
The inner product can be estimated from above as $$ \langle y, Xb\rangle = \langle X^Ty, b\rangle \le \|X^Ty\|_\infty \|b\|_1 \tag3$$ Thus, $\lambda\ge \|X^Ty\|_\infty$ is a sufficient condition for (2) to hold, hence for $b=0$ to minimize.