SVM and quadratic programming

397 Views Asked by At

I wonder if the SVM optimization problem
minimize $||w||^2$ with the contraints $y_i(w^\intercal x_i+b)\ge 1$
could be formulated as a typical quadratic programming problem:
$0.5\cdot z^\intercal Mz$ with $Az\leq d$ and setting z to the vector $\left (\begin{array}{l}w_1\\\ldots\\w_n\\b\end{array} \right )$. The expression $||w||^2$ can be obtained by setting $M$ to $I$.
There is only one problem, $b$ doesnt show up in the objective function. so i would have to introduce a zero line in $M$ but then it isn't invertible any more. Should I take the pseudo-inverse instead? Thanks for any hints.

1

There are 1 best solutions below

0
On

My guess is to to let $z=(w_1,\ldots,w_n,b)^\intercal$, to set $M$ to

$$M = [I_{n\times n} 0_{n\times 1}; 0_{(n+1)\times (n+1)}].$$

and to stack a column of $1$'s in the last column of $A$.

Then solve via a QP solver.