Jensen's inequality with supremum

748 Views Asked by At

Problem

In a paper I am reading now, the author claims that by Jensen's inequality, they have $$ \frac { 1 } { \lambda } \log \exp \left( \lambda \cdot \mathbb { E } _ { \epsilon } \sup _ { h \in \mathcal { H } } \sum _ { i = 1 } ^ { m } \epsilon _ { i } h \left( \mathbf { x } _ { i } \right) \right) \leq \frac { 1 } { \lambda } \log \left( \mathbb { E } _ { \epsilon } \sup _ { h \in \mathcal { H } } \exp \left( \lambda \sum _ { i = 1 } ^ { m } \epsilon _ { i } h \left( \mathbf { x } _ { i } \right) \right) \right)$$ where $\lambda > 0$.

I translate the formula as

$$e^{\lambda \mathbb{E}[\sup_f f]}\leq \mathbb{E}[\sup_f e^{\lambda f}]\tag{1}$$ where $f=\sum_{i=1}^m\sigma_ih(\mathbf{x}_i)$

However, I think this does not work and I can just have $$ e^{\lambda \mathbb{E}[\sup_f f]}\leq\mathbb{E}[e^{\lambda\sup_f f}] $$ and the RHS is actually larger than the RHS of (1) by observing that $f\leq \sup_f f$.

So did I miss something? Any help is appreciated.

1

There are 1 best solutions below

0
On BEST ANSWER

You are not missing much, just the fact that, $\exp$ being increasing,

$$ e^{\sup_{x} \Phi(x)} = \sup_{x} e^{\Phi(x)} $$ for every real-valued family $\{\Phi(x)\}_x$. This holds more generally for any increasing (and continuous) function $f\colon \mathbb{R}\to\mathbb{R}$: $$ f(\sup_{x} \Phi(x)) = \sup_{x} f(\Phi(x)) $$