Using the convexity of the exponential function, how can I show the following bound?

43 Views Asked by At

I'm studying the proof of the Hoeffding inequality proof exposed in "A Probabilistic Theory of Pattern Recognition" page 136 (see https://www.academia.edu/download/51978869/160057889-A-Probabilistic-Theory-of-Pattern-Recognition-Devroye-Gyorfi-Lugosi.pdf).

In order to prove it, the following result is considered :

$$ e^{sx} \leq \frac{x-a}{b-a}e^{sb} + \frac{b - x}{b - a}e^{sa}, $$

where $a\leq x \leq b$ and $s>0$. The paper states that this comes from the fact that the exponential function is convex. I'm trying to show this result using the definition of a convex function and here's what I end up with:

  1. Recall that $f$ is convex on $I$ if for all $x,y \in I$ and all $\lambda \in [0,1]$, the following holds $$ f\left(\lambda x + (1-\lambda)y\right) \leq \lambda f(x) + (1-\lambda) f(y). $$ The function in our case seems to be $$ f(x) = e^{sx}. $$

  2. Since $a \leq x \leq b$, we have $$ \lambda := \frac{x-a}{b-a} \Longrightarrow (1-\lambda) = \frac{b-x}{b-a} $$

  3. Combining previous points yields $$ \text{exp}\left\{\frac{x-a}{b-a}sa + \frac{b-x}{b-a}sb\right\} \leq \frac{x-a}{b-a}e^{sb} + \frac{b - x}{b - a}e^{sa}, $$ where $a$ and $b$ play the role of $x$ and $y$ respectively in the convexity definition.

I would like to continue the reasoning and obtain the desired bound but I don't see how to do it.