Kullback leiber Divergence Proof

Question

Let us consider the distributions $P_1$, $P_2$, $Q_1$ and $Q_2$, then prove verify the following:

$D(P_1 P_2 || Q_1 Q_2) = D(P_1 || Q_1) + D(P_2 || Q_2)$

where $D(P_i||Q_i)$ is the divergence of $P_i$ relative to $Q_i$

Answer 1

With the abuse of notation, we have \begin{align} &D(P_1P_2\parallel Q_1Q_2)\\ &=\sum_{P_1}\sum_{P_2}P_1P_2\log \frac{P_1P_2}{Q_1Q_2} \\ &=\sum \sum P_1P_2\left[\log\frac{P_1}{Q_1}+\log\frac{P_2}{Q_2}\right]\\ &=\sum_{P_2}P_2\sum_{P_1}P_1\log\frac{P_1}{Q_1}+\sum_{P_1}P_1\sum_{P_2}P_2\log\frac{P_2}{Q_2}\\ &=D(P_1\parallel Q_1)+D(P_2\parallel Q_2)~~~~~\square \end{align}