Problem with equation derivation

79 Views Asked by At

I'm studying support vector machines and the book I'm using states in one particular part the following:

We know that: $$\vec{y} = \vec{x} + v\vec{w}$$ and that $$|\;\vec{y}-\vec{x}\;| = 2M$$ and so we can compute that:

$$M = \frac{1}{2\:|\;\vec{w}\;|} = \frac{1}{2\:\sqrt{\vec{w}\cdot\vec{w}}}$$

My question is how did the author get this result? These are the only steps he shows, but I would need more details about what happened when deducing the value for $M$?

Here is a picture which will give more information:

enter image description here

Here is also the part in the book where I get confused, in case it helps (note that the notation is different but idea the same):

enter image description here

Thank you for any help :)

1

There are 1 best solutions below

0
On BEST ANSWER

In SVM, we only consider the margin for the "support" vectors $x$, i.e., those vectors satisfying $w\cdot x = \pm 1$. Thus the margin (distance) is $$\frac{|w\cdot x|}{\|w\|}$$

which is $1/{\|w\|}$. In your case, $y = x + v w$, and both $x$ and $y$ are support vectors on either side. WLOG, suppose $w\cdot x = -1$ and $w\cdot y = 1$. Thus,

$$w\cdot y = w\cdot x + v\|w\|^2$$

$$1 = -1 + v\|w\|^2$$

which gives $v = 2 / \|w\|^2$. On the other hand, $\|y-x\| = v\|w\| = 2M$, we thus have $M = 1/\|w\|$. I don't think $M$ has a $2$ in the denominator, since $v$ already considers the (double) distance from $y$ to $x$, and $M$ is the single sided distance.