Prove logistic loss is convex and lipshitz

54 Views Asked by At

I want to prove that $F(y,y_{pred})=\log(1+e^{-y\cdot y_{pred}})$ is convex while I know that $f(x)=\log(1+e^{-x})$ is convex because $f''(x)>0$, but I cant find a way to apply it on the multivariate case.

Any help will be appreciated.

EDIT: $y\in\{+1,-1\}, y_{pred}\in R$

1

There are 1 best solutions below

2
On BEST ANSWER

If $y$ can only take two values, this is equivalent to just checking two single variable functions for convexity. Call $y_{pred} = x$.

If $y = 1$,

$F(1,x) = f_1(x) = \log(1+\exp(-x))$

If $y = -1$,

$F(-1,x) = f_{-1}(x) = \log(1+\exp(x))$

Since $f_1$ and $f_{-1}$ are both convex (check their second derivatives on their respective domains), $F$ is also convex.