Optimization using Karush-Kuhn-Tucker conditions

296 Views Asked by At

min $y^Tx$

subject to $\|x\|^2 \le 1$

where y is a nonzero vector in $\mathbb R^n$

I rearrange the constraints so that the RHS is $0$.

New constraint: $x_1^2 + \cdots + x_n^2 - 1 = \|x\|^2 - 1 \le 0$

My Lagrangian $\Bbb L = y^Tx + v_i(\|x\|^2 - 1)$

$$\Bbb L' = 0 = y^T + 2v_i(\|x\|)$$

This is where I am a bit stuck so any help would be appreciated

1

There are 1 best solutions below

0
On BEST ANSWER

This problem has a very simple solution not using KKT. We have $$y^Tx = ||x|| \cdot ||y|| \cos\theta$$ where $\theta$ is the angle between $x$ and $y$.

To minimize it pick $x$ s.t. $\cos\theta = -1$ and $||x||=1$ (i.e. $x = -\frac{y}{||y||}$) to get $$\min y^T x = -||y||$$


To solve it using KKT first note that minimizing $y^Tx$ is equal to maximizing

$$L = -y^Tx$$

given $g(x) = ||x||^2 - 1 \leq 0$. This gives us the equations

$$\nabla_i L = \mu \nabla_i g \to x = -\frac{1}{2\mu}y$$

Pluggin it in to $L$ gives

$$L = \frac{1}{2\mu}||y||^2$$

and the condition $g(x) \leq 0$ reads $\frac{1}{4\mu^2}||y||^2 \leq 1 \to \frac{1}{2\mu} \leq \frac{1}{||y||}$ so

$$L = \frac{1}{2\mu}||y||^2 \leq ||y||$$

and we obtain the same solution as above $\min y^Tx = -||y||$ with $x = -\frac{y}{||y||}$.