Showing that a general Hessian matrix is positive semidefinite

260 Views Asked by At

Given vector $a \in \mathbb{R}^n$, show that the scalar field $g : \mathbb{R}^n \to \mathbb{R}$ defined by $$g(\mathbf{x}) = -\log ( f(\mathbf{x}))$$ where $$f(\mathbf{x}) = \dfrac{1}{1+\exp(-a^T\mathbf{x})}$$ is convex.


To show that, we need to show that the Hessian matrix is positive semidefinite, i.e., $\nabla \nabla g(\mathbf{x})\succcurlyeq 0$. I calculated the Hessian as follows:

\begin{equation*} \mathbf{H} = \nabla \nabla g(\textbf{x}) = \left[\dfrac{\partial^2 f(\mathbf{x})}{\partial x_i \partial x_j} \right] = \begin{cases} a_i^2 f^2(\mathbf{x})\exp{(\mathbf{-a}^T\mathbf{x})} & , \ \ \text{if} \ \ \ i = j \\ a_ia_j f^2(\mathbf{x})\exp{(\mathbf{-a}^T\mathbf{x})} & , \ \ \text{if} \ \ \ i \neq j \end{cases} \end{equation*}

I am not sure what is the easiest way to show the positive semidefiniteness of such a functional form of the Hessian. I cannot see how we could show $z^T\mathbf{H}z \geq 0$ for all nonzero $z$ or that the eigenvalues of $\mathbf{H}$ are all non-negative. Any suggestions?

1

There are 1 best solutions below

1
On

You do not need the Hessian. The scalar function $h(s)=\log (1+e^s)$ is convex since its second derivative is non-negative. Your function is the composition of $h$ with the linear function $m(\mathbf{x})=-a^T\cdot \mathbf{x}$ so it is convex as well (compositions of convex and linear functions are convex).