Prove Jensen's inequality?

Question

I was interested to see a proof for Jensen's inequality for the following variant:

Let $X$ be a discrete random variable with finite expected value and let $h:\mathbb{R} \to \mathbb{R}$ be a convex function. then: $h(E[X])\leq E[h(X)]$

Please note, I'm interested in a proof for this variant with a discrete random variable. The proof in Wikipedia doesn't match my needs.

Answer 1

I believe you might be interested in the proof provided in Theorem 2.4 in Probability and Computing: Randomization and Probabilistic Techniques in Algorithms and data Analysis, Second Edition from M. Mitzenmacher and E. Upfal. The only assumption there is that $f$ has a Taylor expansion, so your random variable can be either discrete or continuous.

Let $\mu = \mathbb{E}[X]$. By Taylor's theorem there is a value $c$ such that \begin{align} f(x) &= f(\mu) + f^\prime(\mu)(x-\mu) + \frac{f^{\prime\prime}(c)(x-\mu)^2}{2}\\&\geq f(\mu) + f^\prime(\mu)(x-\mu), \end{align} since $f^{\prime\prime}(c) > 0$. Taking expectations of both sides and applying linearity of expectations and Lemma 2.2 yields the result: \begin{align} \mathbb{E}[f(x)] &\geq \mathbb{E}[f(\mu) + f^\prime(\mu)(x-\mu)] \\&= \mathbb{E}[f(\mu)] + f^\prime(\mu)\mathbb{E}[(x-\mu)] \\&= f(\mu) = f(\mathbb{E}[X]) \end{align}