How to check convexity of a composition when some properties of inner and outer functions are known?

Question

If $g(x)$ function is concave in $x$, and we want $g( f(x) )$ (where $f(x)$ is another function) to be convex in $x$, what are the required properties of $g(x)$ and $f(x)$?

It would be appreciated if you give some references to your answers. Thank you.

Is there a particular rule for this? I think there should be.

---

The outer function is $g(x) = -x\log x$, which is concave in $x$, but I am not able to confirm if the inner function $f(u)$ is convex/concave in u yet. I am looking for some kind of a general rule like "If $f$ and $g$ are convex functions, then so are $m(x) = \max\{f(x),g(x)\}$ and $h(x) = f(x) + g(x)$. (Wikipedia)". In my case, $g$ is concave in $x$ and I am looking for some direct rule that allows me to check (in an easy way) whether the function $g(f(u))$ is either convex or concave in $u$.

Answer 1

There are two simple facts about convexity of composition:

1. If $g$ is convex and $f$ is affine (that is, $f(x)=Ax+b$), then $g\circ f$ is convex.
2. If $g$ is convex increasing, and $f$ is convex, then $g\circ f$ is convex.

Both of these follow quickly from the definition of convexity: for example, the second $$g(f(ta+(1-t)b))\le g(tf(a)+(1-t)f(b)) \le tg(f(a))+(1-t)g(f(b))$$ If one of two functions is not convex, then whatever argument you use will have to quantify its deviation from convexity somehow. In general this is done with the matrix of second derivative (the Hessian). In one-dimensional case the computation is easier: $$ (g\circ f)'' = (g''\circ f)(f')^2+(g'\circ f)f'' $$
It's possible that one of these terms is negative yet is dominated by the other one; but one has to consider concrete functions to say anything more. There are no simple rules for this case.

Some standard references for convex functions:

1. Convexity: an Analytic Viewpoint by Barry Simon
2. Convex Analysis by Ralph Tyrell Rockafellar
3. Notions of Convexity by Lars Hörmander