Entropy: Show that $2[H(X, Y ) + H(Y, Z) + H(Z, X)]\geq 3H(X, Y, Z) + H(X) + H(Y ) + H(Z)$.

44 Views Asked by At

Suppose $X$ , $Y$ and $Z$ are random variables. Show that $2[H(X, Y) + H(Y, Z) + H(Z, X)]\geq 3H(X, Y, Z) + H(X) + H(Y ) + H(Z)$.

I have tried to expand $3H(X,Y,Z)$ but no use. After expanding: $$ H(X|Y) + H(Y|X) + H(Y|Z) + H(Z|Y) + H(X|Z) + H(Z|X) \geq H(Y,Z|X) + H(Z,X|Y) + H(X,Y|Z) $$ I don't know what should I do here. Any idea would be very appreciated. Thanks a lot!

1

There are 1 best solutions below

0
On BEST ANSWER

Using $H(X,Y,Z) = H(X,Y) + H(Z|X,Y)$ and permutations, the question reduces to $$ H(X,Y) + H(Y,Z) + H(Z,X) \overset?\ge H(X|Y,Z) + H(Y|Z,X) + H(Z|X,Y)\\ \qquad \qquad \qquad+H(X) + H(Y) + H(Z).$$

Now expand $H(X,Y) = H(X) + H(Y|X)$ and similar for the rest, and cancel the $H(X) + H(Y) + H(Z)$ term on the RHS to reduce the question to $$H(Y|X) + H(Z|Y) + H(X|Z) \overset?\ge H(Y|Z,X) + H(Z|X,Y) + H(X|Y,Z). $$

But conditioning reduces entropy, so $H(Y|X) \ge H(Y|Z,X)$ et c., so the above is true.