Application of Jensen Inequality for lower bound

Question

In this work Jensen inequality is applied to turn Eq (11) to Eq. (12).

The essential part of Equation 11:

$L = \int q_{X_0}(x_{0}) log (\int q_{X_1,…,X_T | X_0}(x_1,…,x_T|x_{0})*C(x_1,…x_T) d_{x_1},… d_{x_T}) dx_0$

Then Equation 12 states $L \geq \int q_{X_0,...,X_T}(x_0,...,x_T) log (C(x_1,…x_T)) d_{x_1},… d_{x_T}) dx_0$

Here $q$ denotes density functions for the corresponding random variables.

Simply applying Jensen on L results however in an upper bound. Can someone give me a hint how to prove the inequality.

Answer 1

This paper is hard to understand. Still, I am going to use its notation: \begin{align}\tag{9} p\big(x^{(0)}\big)=\int d\mathbf{x}^{(1...T)}\,q\big(x^{(1...T)}\big|x^{(0)}\big)\,p\big(x^{(T)}\big)\prod_{t=1}^T\frac{p\big(x^{(t-1)}\big|x^{(t)}\big)}{q\big(x^{(t)}\big|x^{(t-1)}\big)}\,. \end{align} Presumably the authors integrate here over a cube $[a,b]^T\,.$ Applying Jensen's inequality to

• the finite measure $d\mu=d\mathbf{x}^{(1...T)}\,q\big(x^{(1...T)}\big|x^{(0)}\big)$ on $[a,b]^T$ and to

• the concave function log

yields \begin{align}\tag{*} \log p\big(x^{(0)}\big)\ge\int d\mathbf{x}^{(1...T)}\,q\big(x^{(1...T)}\big|x^{(0)}\big)\,\log\Bigg(p\big(x^{(T)}\big)\prod_{t=1}^T\frac{p\big(x^{(t-1)}\big|x^{(t)}\big)}{q\big(x^{(t)}\big|x^{(t-1)}\big)}\Bigg)\,. \end{align} I think the rest leading to (12) follows from $$\tag{**} \int d\mathbf{x}^{(0)}\,q\big(x^{(0)}\big)\int d\mathbf{x}^{(1...T)}\,q\big(x^{(1...T)}\big|x^{(0)}\big)f\big(\mathbf{x}^{(0...T)}\big)=\int d\mathbf{x}^{(0...T)}\,q\big(x^{(0...T)}\big)f\big(\mathbf{x}^{(0...T)}\big) $$ which should hold for any function $f$. In your case $f$ it is the log of that huge expression in (*).

• Unfortunately, the authors don't seem to define $q\big(x^{(1...T)}\big|x^{(0)}\big)$ somewhere.

Edit.

The authors write \begin{align}\tag{10} L&=\int d\mathbf{x}^{(0)}\,q\big(\mathbf{x}^{(0)}\big)\,\log p\big(\mathbf{x}^{(0)}\big)\,. \end{align} Using (*) and (**) this is larger than $$ \int d\mathbf{x}^{(0...T)}\,q\big(x^{(0...T)}\big)f\big(\mathbf{x}^{(0...T)}\big) $$ where $$ f\big(\mathbf{x}^{(0...T)}\big)=\log\Bigg(p\big(x^{(T)}\big)\prod_{t=1}^T\frac{p\big(x^{(t-1)}\big|x^{(t)}\big)}{q\big(x^{(t)}\big|x^{(t-1)}\big)}\Bigg)\,. $$ In other words, the author's inequality (12) holds.