Prove that $E_S[e^{2(m-1)\Delta (h)^2}] \leq m$

81 Views Asked by At

I'm going through the proof for the PAC-Bayesian inequality, present in the book "Understanding Machine Learning (by Shai Ben-David). The author makes the following claim without stating a proof (it's actually an exercise in the book):

Using Hoeffding's inequality that tells us $$ P_S[\Delta h \geq \epsilon] \leq e^{-2m \epsilon^2} $$

We obtain that $$E_S[e^{2(m-1)\Delta (h)^2}] \leq m$$

How does one proves this result?

1

There are 1 best solutions below

0
On BEST ANSWER

Assuming that $\Delta(h)\ge 0$, \begin{align} \mathsf{E}_S\!\left[e^{2(m-1)\Delta (h)^2}\right]&=\int_0^{\infty}\mathsf{P}_S\!\left(e^{2(m-1)\Delta (h)^2}>t\right)dt \\ &= 1+\int_1^{\infty}\mathsf{P}_S\!\left(\Delta (h)>\sqrt{\frac{\ln(t)}{2(m-1)}}\right)dt \\ &\le 1+\int_1^{\infty}t^{-\frac{m}{m-1}}\,dt=m. \end{align}