Hinge-loss - why 1?

747 Views Asked by At

I can't understand the meaning of 1 in the hinge-loss function: $\max \{0,1-ywx\}$.
What will happen if we use another number, like 5 or 80?
We use the hinge-loss as a loss function of SVM, and the SVM try to find the maximum margin that possible.

Thank you!

1

There are 1 best solutions below

1
On BEST ANSWER

Consider a classic linear classification problem with hinge loss. You want to find $\pmb w$ and $b$, so $y=\pmb w^\intercal \pmb x+b$, and sum of hinge losses $h=\max(0,1-ty)$ is minimal.

However, you can see, that if you found an optimal solution $\pmb w$, $b$, then for a similar problem with hinge loss $h_1=\max(0,80-ty)$, solution $\pmb w_1=80\pmb w$, $b_1=80b$ will be optimal. So one can conclude that this value (1, 80) is just the scaling of the parameter space (and optimization function). It doesn't change the position of the decision border $\pmb w^\intercal \pmb x+b=0$.