Supported Vector Machine constraint condition.

34 Views Asked by At

I think I can follow the following statement, but I can't reach the conclusion "subject to $\sum_{i=1}^n c_i y_0 = 0$". Could someone give me a pointer from which to reach this constraint? Thank you!


Consider a constrained optimisation problem:

For each $i \in \{1,\,\ldots,\,n\}$ we introduce the variable $\zeta_i$, and note that $ \zeta_i = \max\left(0, 1 - y_i(w\cdot x_i + b)\right)$ if and only if $\zeta_i$ is the smallest nonnegative number satisfying $y_i(w\cdot x_i + b) \geq 1- \zeta_i$.

Thus we can rewrite the optimization problem as follows

$ \text{minimize } \frac 1 n \sum_{i=1}^n \zeta_i + \lambda\|w\|^2 $

$ \text{subject to } y_i(x_i \cdot w + b) \geq 1 - \zeta_i \,\text{ and }\,\zeta_i \geq 0,\,\text{for all }i.$

By solving for the Lagrangian dual of the above problem, one obtains the simplified problem

$ \text{maximize}\,\, f(c_1 \ldots c_n) = \sum_{i=1}^n c_i - \frac 1 2 \sum_{i=1}^n\sum_{j=1}^n y_ic_i(x_i \cdot x_j)y_jc_j,$

$ \text{subject to } \sum_{i=1}^n c_iy_i = 0,\,\text{and } 0 \leq c_i \leq \frac{1}{2n\lambda}\;\text{for all }i.$

This is called the dual problem. Since the dual minimization problem is a quadratic function of the $c_i$ subject to linear constraints, it is efficiently solvable by quadratic programming algorithms. Here, the variables $c_i$ are defined such that

$ \vec w = \sum_{i=1}^n c_iy_i \vec x_i.$

https://en.wikipedia.org/wiki/Support_vector_machine#Dual