How to conclude that the following matrix is positive semidefinite or not?

51 Views Asked by At

Fox $w\in \mathbb{R}^{d\times 1}$, define $$f(w)=\frac{1}{n}\sum_{i=1}^n\log(1+\exp(-w^Tx_i))$$ where $x_i\in \mathbb{R}^{d\times 1}$. If I get the Hassian matrix of $f$, that is, $$\nabla^2f(w)=\frac{1}{n}\sum_{i=1}^n\frac{x_i^Tx_ie^{-w^Tx_i}}{(1+\exp(-w^Tx_i))^2}$$

How to know if this matrix is positive semidefinite or not to conclude that $f$ is convex or concave? I do not how to use the definition to check $\nabla^2f(w)$.

1

There are 1 best solutions below

0
On BEST ANSWER

One of the comments suggests that you can prove directly the convexity of $f$ but, since you ask about the matrix $\nabla^2 f(w)$, you can check that, for any $v\in \mathbb{R}^d$, we have: $$ v^\top \, \nabla^2 f(w) \, v = \frac{1}{n}\sum_{i=1}^n\frac{v^\top x_i x_i^\top v \,e^{-w^Tx_i}}{(1+\exp(-w^Tx_i))^2} = \frac{1}{n}\sum_{i=1}^n\frac{<x_i, v>^2 \,e^{-w^Tx_i}}{(1+\exp(-w^Tx_i))^2} \ge 0 $$ since it is a sum of non-negative terms.