The problem of support vector machine - How to minimize $||w||^{2}$ subject to constraints of the form $\alpha w_{1}+\beta w_{2}+b\geq\pm1$

36 Views Asked by At

I am studying the subject of support vector machines from an online course.

I am given four points and their classification $$ x_{1}=((5,4),+),\, x_{2}=((8,3),+) $$

$$ x_{3}=((3,3),-),\, x_{4}=((7,2),-) $$

I was told to find take a pair of points with the same classification, and find the best maximum-margin linear classifier that is parallel to the line connecting those points.

In this case there are two classifiers to find, and I am trying to find the one that correspond to the set of positive points.

The end goal is to minimize $$ ||w||=w_{1}^{2}+w_{2}^{2} $$

under the constraint that $$ \langle x_{i},w\rangle+b\geq1 $$

for $i=1,2$ and $$ \langle x_{i},w\rangle+b\leq-1 $$

for $i=3,4$.

My progress:

The line connecting the positive points have a slope of $$ \frac{3-4}{8-5}=-\frac{1}{3} $$

so, from my understanding, $w$ have a slope of $$ -\frac{1}{-\frac{1}{3}}=3 $$

and so if $w\in\mathbb{R}^{2}$ is denoted by $w=(w_{1},w_{2})$ I get $\frac{w_{2}}{w_{1}}=3$, hence $w=(w_{1},3w_{1})$ and I need to minimize $||w_{1}||^{2}$ under the constraints that $$ 5w_{1}+4w_{2}+b\geq1 $$ $$ 8w_{1}+3w_{2}+b\geq1 $$

$$ 3w_{1}+3w_{2}+b\leq-1 $$ $$ 7w_{1}+2w_{2}+b\leq-1 $$

and substituting $w_{2}=3w_{1}$ the constraints are $$ 17w_{1}+b\geq1 $$

$$ 12w_{1}+b\leq-1 $$ $$ 13w_{1}+b\leq-1 $$

(the first two inequalities translate after the substitution to the same constraint)

How do I go at solving this optimization problem ?