Prove the equality: $H_{\tau_1, \tau_2} + \mathcal{H}(p(Q|\tau_1, \tau_2)) = \mathcal{H}(p(Q))$

25 Views Asked by At

Let $Q$ be a random variable and $\tau$ be some parameters. Let us define a summation of KL divergences $$H_{\tau_1, \tau_2} = D_{KL}(p(Q|\tau_1, \tau_2)\Vert p(Q|\tau_1)) + D_{KL}(p(Q|\tau_1)\Vert p(Q)) $$

It is said that the following equality holds $$H_{\tau_1, \tau_2} + \mathcal{H}(p(Q|\tau_1, \tau_2)) = \mathcal{H}(p(Q))$$ where $\mathcal{H}(\cdot)$ is the entropy.

I am quite confused about how to prove the validity of the equality above.