Consider the following LASSO problem $min_{\beta} \sum\limits_{i=1}^n(y_{i}-\sum\limits_{j=1}^p x_{ij}\beta_{j})^2$ , subject to $\sum\limits_{j=1}^p|\beta_{j}|\leq t$
where $t \geq 0$ is a constant.
(a) If $t = 0$, compute $\hat{\beta}_{j}^{lasso}$ , for $j = 1, . . . , p$.
(b) Define $t_{0}= \sum\limits_{j=1}^p \hat |{\beta}_{j}^{ols}| $. Prove that, if $t \geq t_{0}$, then
$\hat{\beta}_{j}^{lasso} $=$ \hat{\beta}_{j}^{ols}$
My approach: for (b) $ \hat{\beta}_{j}^{ols}=\sum\limits_{i=1}^n(y_{i}-\hat{y})^2$ and $\hat{\beta}_{j}^{lasso} = \sum\limits_{i=1}^n(y_{i}-\hat{y})^2 + \lambda \sum\limits_{j=1}^p |\beta_{j}|$,
when $\lambda =0 $ we can get get $\hat{\beta}_{j}^{lasso} $=$ \hat{\beta}_{j}^{ols}$
I am not sure about the approach, also don't know how to do part (a). Please guide.
Are you a student at UA? Let me give you some hint.
Consider the problem below.
Find real number x and $y$ which minimize $x^2 + y^2$ subject to $|x|+|y|\le t$.
(a)If t = 0, find the x and $y$.
(b)If t = 5, find x and $y$.
For part (a), the restriction is $|x|+|y|\le 0$, so x must be 0 and $y$ must be 0. Otherwise $|x|+|y|> 0$.
For part (b), the global solution for minimizing $x^2 + y^2$ without any restriction is x=0, $y$=0. This global solution without any restriction satisfied the restriction $|x|+|y|\le 5$. Thus, the solution for minimize $x^2 + y^2$ subject to $|x|+|y|\le 5$ is also x=0, $y$=0.
Please work on your question with the hint.