Lagrange Multiplier 0 in SVM

Question

In the equation of SVM

$$ L(x)=f(x)-\sum \alpha g(x) $$

The KKT condition states that $$\alpha>=0$$

but what I would like to understand is, what's the meaning of $$\alpha=0$$ in SVM case?

Answer 1

I assume your optimization problem is like $$\begin{eqnarray}\mathop{\text{min}}_{x}\quad&f(x)& \\ \text{subject to }\quad&g_i(x)&\ge0,\quad i=1,\cdots,m \end{eqnarray}$$ Hence the Lagrangian associated with the problem is as $$L(x,\alpha)=f(x)-\sum_{i=1}^m\alpha_ig_i(x)$$

And your dual problem for the primal one is as $$\mathop{\text{max}}_{\alpha\ge0}\;G(\alpha)$$ where $G$ is defined by $$G(\alpha)\equiv\mathop{\text{min}}_{x}\; f(x)-\sum_{i=1}^m\alpha_ig_i(x)$$

---

For $\alpha_i=0$ ? No it tells you nothing. But if $\alpha_i \gt 0$, it means $g_i$ is satisfied, i.e. $g_i(x)=0$

In SVM (hard margin) case, $g_i$ is defined as "wheather the $i$th instance touches the boundary" (there are many versions of mathematical definition so I'd rather use a sense). So $\alpha_i\gt0$ implies $x_i$ touches the boundary, so called "support vector".

If $\alpha_i=0$, then $x_i$ may or may touch the boundary. But generally we still call that a non-support vector.