Prove that that $H(X,Y) + H(Y,Z) + H(Z,X) \geq 2H(X,Y,Z)$

Question

Given that $X,Y,Z$ are random variables. Prove that that $H(X,Y) + H(Y,Z) + H(Z,X) \geq 2H(X,Y,Z)$

I know that $$H(Y,Z) \geq H(Y,Z|X) = H(Y|Z,X) + H(Z|X) \geq H(Y|Z,X) + H(Z|X,Y)$$ but I'm stuck here. And prior ideas would be a great help! Thanks a lot

Answer 1

Expand the RHS like this

$$2H(X,Y,Z) = H(X,Y,Z) + H(X,Y,Z) = \left[ H(X)+H(Y|X)+H(Z|X,Y) \right]\\ +\left[ H(Y)+H(Y|Z)+H(X|Y,Z) \right]$$

Now expand the LHS using $H(X,Y) = H(X)+H(X|Y), H(Y,Z) = H(Y)+H(Y|Z)$ and then simplify the common terms from both sides to arrive at this

$$H(X,Z) \ge H(Z|X,Y) + H(X|Y,Z) \\ \Rightarrow H(X) + H(Z|X) \ge H(Z|X,Y) + H(X|Y,Z)$$

But we know that $H(X) \ge H(X|Y,Z)$ (if $X$ is independent of $Y,Z$ the equality attains) and also $ H(Z|Y) \ge H(Z|X,Y)$ (in this case equality is attained if $Z|Y$ is independent of $X$).