Upperbound of Kullback-Leibler between marginals

Question

I want to know if for every pair of $p(x, y)$ and $q(x, y)$ the following inequality $$ \int_X p(x) \log \frac{p(x)}{q(x)} dx \leq \int_X\int_Y p(x, y) \log \frac{p(x, y)}{q(x, y)}dxdy $$ is valid. I tried unsuccessfully to use Jensen's and Gibbs' inequalities.

Answer 1

I think I found the solution.

$$ p(x)\log\frac{p(x)}{q(x)} \leq \int_Yp(x,y)\log\frac{p(x,y)}{q(x,y)}dy $$

implies

$$ \int_X p(x)\log\frac{p(x)}{q(x)}dx \leq \int_X \int_Yp(x,y)\log\frac{p(x,y)}{q(x,y)}dydx $$.

Therefore, $$ \int_Y p(x, y) \log \frac{p(x,y)}{q(x,y)}dy \\ = p(x) \int_Y p(y | x) \log \frac{p(x,y)}{q(x,y)}dy \\ = p(x) \int_Y p(y | x) \log \frac{p(y|x)p(x)}{q(y|x)q(x)}dy \\ = p(x) \int_Y p(y | x) \bigg(\log \frac{p(y|x)}{q(y|x)} + \log \frac{p(x)}{q(x)}\bigg)dy \\ = p(x) \int_Y p(y | x)\log \frac{p(y|x)}{q(y|x)}dy + p(x) \int_Y p(y | x) dy \log \frac{p(x)}{q(x)} \\ = p(x) \int_Y p(y | x)\log \frac{p(y|x)}{q(y|x)}dy + p(x) \log \frac{p(x)}{q(x)}. $$ since $$ p(x)\int_Y p(y | x)\log \frac{p(y|x)}{q(y|x)}dy $$ is always positive, then $$ p(x) \log \frac{p(x)}{q(x)} \leq \int_Y p(x, y) \log \frac{p(x,y)}{q(x,y)}dy $$ which implies, as already mentioned, $$ \int_X p(x)\log\frac{p(x)}{q(x)}dx \leq \int_X \int_Yp(x,y)\log\frac{p(x,y)}dx{q(x,y)}dy. $$