Can we swap integral and limit?

91 Views Asked by At

$$ D_{\alpha}(p \| q)=\frac{\int ( \alpha p(x)+(1-\alpha) q(x)-[p(x)]^{\alpha}[q(x)]^{1-\alpha}) d x}{\alpha(1-\alpha)},\;\alpha\in(0;1).$$ Here $p(x)$ and $q(x)$ are probability densities functions. It's easy to calculate using L’Hospital’s rule that $\lim\limits_{\alpha\to 0} D_{\alpha}(p \| q) = KL(q||p)$. $KL(q||p)$ is Kullback–Leibler divergence. But my question is can we swap integral and limit? I solved this task when this operation(swapping) is allowed. I tried to apply Levy's theorem but unsuccessfully.

1

There are 1 best solutions below

0
On BEST ANSWER

Given that the integrand is non-negative, you have that for all $\alpha\in[0,1]$ and $x\in \mathbb R$ \begin{align} \left|\alpha p(x)+(1-\alpha) q(x)-[p(x)]^{\alpha}[q(x)]^{1-\alpha}\right| &=\alpha p(x)+(1-\alpha) q(x)-[p(x)]^{\alpha}[q(x)]^{1-\alpha}\\ &\le \alpha p(x)+(1-\alpha) q(x)\tag1\\ &=\alpha(p(x)- q(x)) + q(x)\\ &\le \max\{p(x),q(x)\}\tag2 \end{align} Where $(1)$ is a consequence of the non-negativity of $p(x)^\alpha q(x)^{1-\alpha}$.

As for inequality $(2)$, it follows from observing that the map $\varphi:\alpha\mapsto \alpha(p(x)- q(x)) + q(x)$ is monotone and thus reaches its extremal values at the endpoints of $[0,1]$.

We thus conclude that the family of functions $(f_\alpha)_{0<\alpha<1}$ defined by $$f_\alpha :x\in\mathbb R\mapsto \alpha p(x)+(1-\alpha) q(x)-[p(x)]^{\alpha}[q(x)]^{1-\alpha}$$ Is dominated by an integrable function and the dominated convergence theorem then allows us to swap integral and limit, as desired.