Support Vector Machines (SVMs), unclear math steps

83 Views Asked by At

I am studing the maths behind the Support Vector Machines (SVMs), but there are two not clear steps.

Following the video of 16. Learning: Support Vector Machines (MIT OpenCourseWare, minutes 14:24), we have the following steps

$Width = (\bar{x}_{+}-\bar{x}_{+}) \cdot \frac{\bar{w}}{\left \| w \right \|} = \frac{(1-b)(b-1)}{\left \| w \right \|} = \frac{2}{\left \| w \right \|}$

then

$Max = \frac{1}{\left \| w \right \|}$

therefore

$Min = \left \| w \right \| = \frac{1}{2}\left \| w \right \|^2$

I don't understand (1) why max is $\frac{1}{\left \| w \right \|}$ and min is $\left \| w \right \|$, and (2) why $\left \| w \right \|$ = $\frac{1}{2}\left \| w \right \|^2$

1

There are 1 best solutions below

3
On BEST ANSWER

You are putting "equals signs" where there should not be any. (It is not "$\max = \frac{1}{\|w\|}$" he means "$\max \frac{1}{\|w\|}$" i.e. "maximize the quantity $1/\|w\|$.")

If the goal of SVM is to maximize the width, which is shown to be $2/\|w\|$, then it is equivalent to minimize $\|w\|$. (Choosing $w$ to make $2/\|w\|$ big will also make $\|w\|$ small.)

If you want to minimze $\|w\|$, it is equivalent to minimizing $\frac{1}{2}\|w\|^2$ (making $\|w\|$ small will make $\frac{1}{2} \|w\|^2$ small). The particular choice of $\frac{1}{2} \|w\|^2$ is convenient for steps that appear later.