In the formulation of SVM, we need to compute the margin between an arbitrary point $x^{(i)}$ in the N -dimensional space and a hyperplane $w^T x + b = 0$, which can be formulated as the following optimization problem:
$$ \min_{x} ||x^{(i)}-x||_2 $$
$$ s.t. \,\,\, w^Tx+b=0 $$
I think that, geometrically, optimal should be lying on the hyperplane and be a projection of $x^{(i)}$ on it. But not sure how to derive it in optimization. I am new to optimization, so would like to ask if my thoughts about it are correct:
- Can problem reformulated as just $\min_{x}{x^{(i)}-x}$ as l2 norm is >=0?
- If it cannot be rewritten then Lagrangian of above is: $L(x, \nu)=||x^{(i)}-x||_2+\nu(w^Tx+b)$?
- To proceed further I need to find the optimal solution for $L(x, \nu)$. But I am not sure how p* or d* are related to this problem. I mean I understand, what are those, but how they will help me to find optimal?
Yes, this is homework, so do not put answers for problem itself. Thank you.