How to remember which direction Jensen's inequality is in?

Question

For a real valued random variable and a convex function $f$, Jensen's inequality is given by: $$f(\mathbb E X) \leq \mathbb E f(X)$$

How do you remember which direction the inequality is in? I always have to think about how for convex functions, the chord connecting two points on the graph is always above the graph itself.

Answer 1

For this and other variants of this inequality, I memorize it by asking in which side do you evaluate more times the function $f$, then that side is greater (or equal).

Answer 2

Very informally, a function $f$ is convex if the value of $f$ at the average of two points lies below the average of the values at the two points. In symbols, $f\left(\frac{x + y}{2}\right) \leq \frac{1}{2} \left(f(x) + f(y)\right)$. However, it's very good to experiment with drawing what a continuous function satisfying this inequality can possibly look like. The version you have above is a direct generalization, where "average of two points" is replaced by "expectation." Both are notions of averaging, hence it's not so hard to build a mental association between the midpoint and expectation versions.

In fact, $f(\mathbb{E}(X)) \leq \mathbb{E}(f(X))$ for all random variables $X$ --- on a sufficiently rich probability space (if you want, this is "legalese") --- is equivalent to convexity of $f$.

Now you could ask: how to remember $f\left(\frac{x + y}{2}\right) \leq \frac{1}{2} \left(f(x) + f(y)\right)$ as opposed to $f\left(\frac{x + y}{2}\right) \geq \frac{1}{2} \left(f(x) + f(y)\right)$? Well, in a way, there is no way to remember which one is convex since the other one is concave and the distinction is, after all, arbitrary --- as naming conventions always are. (Here it's unfortunate that most American students have it beaten into them that these notions are "concave up" and "concave down," only to see "convex" and "concave" later in life and have to rethink these concepts.)

At any rate, with some experience, you learn that convex means "bending upwards" or I just think "convex functions have local minima, but not local maxima" and that helps with distinguishing between convex and concave. If you think "concave = bending down" and "convex = bending up" (or "convex = upwards parabola" versus "concave = downwards parabola"), only one of these mental pictures coincides with $f\left(\frac{x + y}{2}\right) \leq \frac{1}{2} \left(f(x) + f(y)\right)$.