Relative Entropy on a countably infinite alphabet

29 Views Asked by At

Suppose $P,Q$ are two PMFs on a countably infinite alphabet $\mathcal{A}$, and define the relative entropy by $$D(P||Q) = \sum_{x\in\mathcal{A}}P(x)\log_2\frac{P(x)}{Q(x)}.$$

Suppose that $D(P||Q)<\infty$. I need to show that $$\sum_{x\in\mathcal{A}}P(x)\left|\log_2\frac{P(x)}{Q(x)}\right|<\infty,$$ but I really don't know how to do this. Any advice would be greatly appreciated!