Is there a concrete proof showing that hinge loss is an upper bound on the 0-1 loss?

Question

While it is stated in several places that the Hinge loss is a convex upper bound on the 0-1 loss, is there a proof behind it? From what I have seen, most resources just show the plots of hinge loss and 0-1 loss and mention that its a convex upper bound on the 0-1 loss. For y>-1 and a small value of y_pred, wont the hinge loss be smaller than 0-1?

Plot of Hinge loss and 0-1 loss

Answer 1

There are several functions called the "hinge loss" and depending on which one you pick, the answer may vary... but from your graph it looks like you're considering $h(x)=\max\{0,1-x\}$. If this is the case, then yes, it is larger than $1_{\mathbb{R}_-}$

Proof: for any $x\in\mathbb{R}$

\begin{aligned} h(x)&= \begin{cases} 1-x &\text{if } x\leq 1\\ 0 &\text{if } 1<x \end{cases}\\ &= \begin{cases} 1-x &\text{if } x<0\\ 1-x &\text{if } 0\leq x\leq 1\\ 0 &\text{if } 1 < x \end{cases}\\ &\geq \begin{cases} 1 &\text{if } x<0\\ 0 &\text{if } 0\leq x\leq 1\\ 0 &\text{if } 1 < x \end{cases}\\ &=1_{\mathbb{R}_-} \end{aligned}