KKT conditions for L2 norm

838 Views Asked by At

I have the following minimization problem

\begin{gather*} \text{minimize} \quad ||w|| \quad \quad w\in\mathbb{R}^2 \\ \text{subject to} \quad w_1+w_2+1\le0 \end{gather*}

I would like to solve it and check for KKT conditions. I'm not entirely sure of the procedure. Does one guess for solutions then check for KKT or is there an analytical way of doing it?

Afaik the conditions are

$$\frac{\partial L}{\partial w_1}=\frac{w_1}{||w||}+\mu=0$$ $$\frac{\partial L}{\partial w_2}=\frac{w_2}{||w||}+\mu=0$$ $$\mu(w_1+w_2+1)=0$$ $$ w_1+w_2+1\le0$$ $$\mu \ge 0$$

2

There are 2 best solutions below

0
On BEST ANSWER

We find the solution by solving the KKT conditions. In particular, observe that $$\frac{w_1}{\|w\|}+\mu = 0 \implies \frac{w_1}{\|w\|} = -\mu = \frac{w_2}{\|w\|} \implies w_1=w_2.$$ Also, $\|w\| \neq 0$ as $\begin{bmatrix}0 \\ 0\end{bmatrix}$ does not lie in the feasible region. Consequently, $$\mu = -\frac{w_1}{\|w\|}=-\frac{w_2}{\|w\|}\neq 0.$$ As a result, $$w_1+w_2+1 = 0 \implies w_1=w_2=-1/2.$$ So the optimal solution is $w^* = -\begin{bmatrix}1/2 \\1/2\end{bmatrix}$, and the minimum value of the function is $1/\sqrt{2}$.

0
On

@MathLover answer is the right derivation.

Yet, since this problem is in $ \mathbb{R}^{2} $ one could even draw it:

enter image description here

Basically drawing the line $ {w}_{1} + {w}_{2} = -1 $.
The valid domain is below it.
Then drawing the $ {L}_{2} $ Ball with radius $ \alpha $.
Setting $ \lambda = 0 $ and then making it grow until it intersect the valid domain at first.
The first intersection point is the solution.

The $ \lambda $ from the above is the same one from the KKT Conditions.
Namely it is the one which obeys all conditions.