Convexity of logistic loss

7.8k Views Asked by At

How to prove that logistic loss is a convex function?

$$f(x) = \log(1 + e^{-x})?$$

I tried to derive it using first order conditions, and also took 2nd order derivative, though I do not see neither $f(y) \geq f(x) + f'(x)(y-x)$, nor positive definiteness (aka always positive second derivative in this case).

2

There are 2 best solutions below

4
On BEST ANSWER

$$f'(x) = \frac{-\exp(-x)}{1+\exp(-x)}=-1+(1+\exp(-x))^{-1}$$

$$f"(x) = -(1+\exp(-x))^{-2}(-\exp(-x))=\frac{\exp(-x)}{(1+\exp(-x))^2} > 0$$

0
On

You can simplify the given function and then take $2$nd order derivative: $$y=\ln{(1 + e^{-x})}=\ln{\frac{e^x+1}{e^x}}=\ln{(e^x+1)-x}.$$ $$y'=\frac{e^x}{e^x+1}-1,$$ $$y''=\frac{e^x(e^x+1)-e^{2x}}{(e^x+1)^2}=\frac{e^x}{(e^x+1)^2}>0.$$