LASSO: Dual form problem.

195 Views Asked by At

I would like to know the procedure, step by step, to arrive at the dual form of the optimization problem for lasso:

From: \begin{equation}\label{ridge2} \hat{\beta}^{lasso}=\min_{\beta}{\sum_{i=1}^{n}\left(y_{i}-\beta_{0}-\sum_{j=1}^{p}\beta_{j}X_{ij}\right)^2} \quad \mathrm{s.a} \quad \sum_{j=1}^{p}|\beta_{j}| \leq t \end{equation}

To: \begin{equation} \hat{\beta}^{lasso}=\min_{\beta} \left\lbrace \sum_{i=1}^{n}\left(y_{i}-\beta_{0}-\sum_{j=1}^{p}\beta_{j}X_{ij}\right)^2+\lambda\sum_{j=1}^{p}|\beta_{j}| \right\rbrace \end{equation}

In all the books I have seen they say that the two forms are equivalent, using the Lagrange multipliers, but I don't know why. That part is always omitted. It may be trivial, but I don't understand it.