duality gap of LASSO -- the idea behind residual rescaling

126 Views Asked by At

I'm reading through this paper, where they find the dual of LASSO and then find a lower bound of the optimal solution of LASSO. Let $p^*, q^*$ be the optimal solution of LASSO and its dual consequently. Let us call the dual problem $G(\nu)$, we know that $G(\nu)<p^*$. To find the duality gap (i.e., the sub-optimal) we need to find the tightest bound (i.e., the maximum value of $G(\nu),\nu$ is a dual feasible). However, the authors pick a $\nu$ that, from my understanding, is not the optimal value of $G(\nu)$. Although it is simple to calculate, I think it is not understandable heuristic. The following screenshot shows it (taken from the top of the right column in page 4). I have tried to find a closed form solution of the dual problem but what I got is not even close to this. What is the intuition behind such a heuristic? (in case it is indeed a heuristic.)

Edit: This heuristic is called residual rescaling but I was not able to find the original paper that proposed it. So, I am asking about the intuition behind it?

enter image description here