Proving concavity of a function in multiple variables

Question

How do I prove that $f(\vec{x}) = \vec{b}.\vec{x} - \log \left (1+e^{\vec{a}.\vec{x}} \right )$ is concave? where $\vec{a}$ and $\vec{b}$ are constant vectors.

My steps are as follows: The first term is a linear function in $\vec{x}$ so the first term is concave in $\vec{x}$.

If I can prove that the second term is concave in $\vec{x}$, then the sum of two concave functions is concave.

In order to show that the second term is concave, I have to show that $\log \left (1+e^{\vec{a}.\vec{x}} \right )$ is convex, right? How to do that?

Answer 1

Let $f(x) = \log(1 + e^{a^\top x})$. It suffices to show that the Hessian is positive semi-definite.

The gradient is $$\frac{1}{1+e^{-a^\top x}} a$$ and the Hessian is therefore $$\frac{e^{-a^\top x}}{(1+e^{-a^\top x})^2} a a^\top.$$

The Hessian is a rank one matrix whose only nonzero eigenvalue is $\frac{e^{-a^\top x}}{(1+e^{-a^\top x})^2} \|a\|^2 \ge 0$, so the Hessian is positive semi-definite.

Answer 2

Differentiate the function $x \mapsto \log(1+e^x)$ twice to show that it is strictly convex.

If $f$ is convex and $g$ is affine then it is straightforward to show that $f \circ g$ is convex.

The map $x \mapsto \langle a, x \rangle$ is linear hence affine.