Is the trace of the Hessian matrix of the logistic loss function a convex function?

188 Views Asked by At

Consider the logistic loss function

$$\ell(x, y,w) = \log \left( 1 + \exp \left(- y w^T x \right) \right)$$

where $x \in \Bbb R^d$ is an input sample and $y \in \{0,1\}$ is its label. We know that logistic loss is convex w.r.t model parameter $w$, so the Hessian matrix $H = \nabla_{w}^2 \ell(x,y,w)$ is positive semidefinite.

Is $\mbox{trace}[H]$ also a convex function w.r.t $w$? Can we generalize this result for any convex loss function?

1

There are 1 best solutions below

0
On

No, even in the simple case of $d=x=y=1$, $H = e^w / (1+e^w)^2$ is not convex (just plot it).