Solving SVM Lagrangian dual

93 Views Asked by At

With the Lagrangian dual equal to:

$$-\frac{1}{2} \sum_{i=1}^n\sum_{j=1}^n\alpha_{i}\alpha_{j}y_{i}y_{j}x_{i}\cdot x_{j} + \sum_{i}^n \alpha_{i}$$ and $$w = \sum_{i=1}^n \alpha_{i}y_{i}x_{i},$$ $$\sum_{i=1}^n \alpha_{i} y_{i} = 0$$

at the optimal values for w and b, is it possible to solve for the alphas by solving the system of equations created by doing the following:

1) Calculating the partial derivatives of the Lagrangian dual with respect to each alpha, and setting all of these partial derivatives equal to zero.

2) Making use of the fact that: $$\sum_{i=1}^n \alpha_{i} y_{i} = 0$$ at the optimal solution.

I created a coefficient matrix from the above system of equations and solved for the alphas making used of Gauss-Jordan elimination, however this method only proved to work for certain examples and failed for others.

If this is a poor approach to solving this problem could you please advise me on what I could do to solve the problem without making use of QP solver. Thanks very much.