I was reading this blogpost which talks about the properties of binary entropy function.
It was stated that $(1-p q) h\left(\frac{p-p q}{1-p q}\right)=h(p)+p h(q)-h(p q) \quad 0 \leq p \leq 1,0 \leq q \leq 1, p q<1$
I am unable to find this identity elsewhere. I am interested in understanding the proof of this identity. Can someone provide a reference proving this? If not, can someone provide the proof for it?
Also, could someone provide an intuition or meaningful interpretation for this property?
Here's an interpretation.
Recall this property: the entropy of a non overlapping mixture of distributions equals the mixture (weighed average) of the individual entropies, plus the entropy of the mixing factor (binary entropy).
Now, consider a random variable $X$ taking $3$ values with probabilities $${\bf p} = [a; b;c] \tag 1$$
with $a+b+c=1$.
This can be considered as a mixture of two non-overlapping distributions thus: $$ a [1; 0; 0] + (1-a) \left[ 0; \frac{b}{1-a} ; \frac{c}{1-a}\right] \tag 2 $$
Hence $$H(X) = a 0 + (1-a) h\left(\frac{b}{1-a} \right) + h(a) = (1-a) h\left(\frac{b}{1-a} \right) + h(a) \tag3 $$
But $(1)$ it can also be decomposed in this way:
$$ b [0; 1; 0] + (1-b) \left[ \frac{a}{1-b} ; 0; \frac{c}{1-b}\right] \tag 4 $$
Hence $$H(X) = (1-b) h\left(\frac{a}{1-b} \right) + h(b) \tag5 $$
or
$$ (1-b) h\left(\frac{a}{1-b} \right) + h(b) = (1-a) h\left(\frac{b}{1-a} \right) + h(a) \tag 6$$
Letting $1-a=p$ and $ b = pq$ we get the original equation.