KL-divergence of two distributions using probability measure

68 Views Asked by At

I am reading an article stated that:

Given $\mu$ is a probability measure, a measurable set $A$, and $\hat{\mu}(\cdot) = \mu(\cdot \bigcap A)$, then $D_{KL}(\hat{\mu}||\mu) = - \log \mu(A)$.

How do they get this result? I worked around with the KL divergence but cannot get this.

1

There are 1 best solutions below

4
On BEST ANSWER

Clearly $\hat \mu\ll \mu$; and since $\hat \mu(X) = \mu(X)/\mu(A)$ for any measurable $X\subset A$, the Radon-Nikodym derivative $d\hat \mu/d\mu = 1/\mu(A)$. Thus $$D_{KL}(\hat \mu||\mu) = \int_A \log \frac{d\hat \mu}{d\mu} = -\log \mu(A).$$