$ \def\KL#1#2{\operatorname{D_{KL}}(#1 \| #2)} \def\E{\operatorname*{\mathbb{E}}} \def\dee{\mathop{\mathrm{d}\!}} $
I'm trying to convince myself that the following identity holds:
$$\KL{P}{R} = \KL{P}{Q} + \E_{x\sim P}\left[\log\frac{q(x)}{r(x)} \right] $$
where $P,Q,R$ are probability measures on $\mathbb R$, and $q,r$ are the densities of $Q,R$, respectively. Derivation below.
This identity seems to hold if $P\ll Q$ and $P\ll R$ (with notation '$\ll$' meaning 'absolutely continuous with respect to'). My questions are: is it true that this requirement is sufficient? Is this requirement necessary?
Derivation:
To be explicit here, densities $p,q,r$ are the the RN derivatives of $P,Q,R$ respectively, each with respect some common measure $\nu$, which we could take to be the Lebesgue measure or counting measure.
$$ \begin{aligned} \KL{P}{R} &= \int_{-\infty}^{\infty}p(x)\log\frac{p(x)}{r(x)} \dee{\nu(x)}\\ &= \int_{-\infty}^{\infty}p(x)\log\frac{p(x)q(x)}{q(x)r(x)} \dee{\nu(x)}\\ &= \int_{-\infty}^{\infty}p(x)\log\frac{p(x)}{q(x)} \dee{\nu(x)} + \int_{-\infty}^{\infty}p(x)\log\frac{q(x)}{r(x)} \dee{\nu(x)} \\ &= \KL{P}{Q} + \int_{-\infty}^{\infty}p(x)\log\frac{q(x)}{r(x)} \dee{\nu(x)} \end{aligned} $$
In order for this derivation to work, I think I need to assume
- $P\ll R$ (definition of KL, in the first line)
- $P\ll Q$ (so that $p(x)/q(x)$ is defined in the second line)
Is this correct, or am I missing something? Do I need to make any other assumption? In particular does this requires that $Q\ll R$?