Support vector machine with the nonseparable case & SMO algorithm

27 Views Asked by At

I am self - studying about support vector machine. Because I want to understand clearly about the algorithm, I tried to solve some simple exercises by hand.

My problem: Consider two classes of data, where three points $(0.5,0),(2,1),(1,1)$ have the label $+1$ and two points $(1,0),(0,1)$ have the label $-1$. Let's choose $C=5$ .Compute the maximum margin hyperplane that separates the two classes of data.
My attempt The dual problem of my problem is $$ \begin{array}{ll} & \max \sum_{i=1}^5 \alpha_i-\frac{1}{2} \sum_{i, j=1}^5 \alpha_i \alpha_j y_i y_j\left\langle\mathbf{x}_i, \mathbf{x}_j\right\rangle \\ \text { s.t. } & 0 \leq \alpha_i \leq C, \forall i=1, \ldots, 5, \\ & \sum_{i=1}^5 \alpha_i y_i=0 . \end{array} $$ where the Hessian matrix $H=\left(H_{i j}\right)_{i, j=1, \ldots, 5}\left(H_{i j}=y_i y_j\left\langle\mathbf{x}_i, \mathbf{x}_j\right\rangle\right)$ is defined as $$ H=\left[\begin{array}{rrrrr} 0.25 & 1 & 0.5 & -0.5 & 0 \\ 1 & 5 & 3 & -2 & -1 \\ 0.5 & 3 & 2 & -1 & -1 \\ -0.5 & -2 & -1 & 1 & 0 \\ 0 & -1 & -1 & 0 & 1 \end{array}\right] $$ From here I want to find the solution of dual problem by hand. It's time consuming if I used the KKT conditions (because there are a lot of variables).
I intend to use to SMO algorithm (which were mentioned on Andrew lecture note), but I don't understand this algorithm clearly.
So can you help me to use SMO algorithm to solve the problem? How many iterations do I need to do?