Prove an inequality concerning Kullback-Leibler Divergence

Question

For any distribution $P$ and $Q$ on $\mathcal{X}$ and any function $f:\mathcal{X} \rightarrow \mathbb{R}$, prove the following inequality: $$\mathbb{E}_{x\sim Q}[f(x)]\le \ln \mathbb{E}_{x\sim P}[\exp(f(x))]+KL(Q||P)$$ I have no idea on transforming the expectation to a Kullback-Leibler Divergence at all. Is there a simple proof on the inequality (for example, just using the knowledge of probability theory and calculas)? Thank you!

Answer 1

This computation appears in the study of variational inference. Note that $$ \begin{align} \log \mathbb{E}_{x\sim P}(f(x)) &\triangleq \log \int_x p(x) f(x) dx\\ &= \log \int_x q(x) \frac{p(x)}{q(x)}f(x) dx\\ &\geq \int_x q(x) \log \left(\frac{p(x)}{q(x)}f(x) \right)dx\\ &= \int_x q(x) \log f(x) dx +\int_x q(x) \log \left(\frac{p(x)}{q(x)} \right)dx\\ &\triangleq \mathbb{E}_{x\sim Q}(\log(f(x)))-KL(Q||P), \end{align} $$