Proof of inequality with entropy

166 Views Asked by At

I can not prove this inequality $2H(a,b,c) \leq H(a,b) + H(a,c) + H(b,c|a),\ H-entropy $. I tried do it by using chain rule and this inequality $H(X|Y) \leq H(X;Y)$ but without any success. Please help me to understand how to prove this type of inequalities. Thank you in advance.

1

There are 1 best solutions below

0
On BEST ANSWER

$$H(a,b,c) = H(a)+H(b,c\mid a) \tag{1}$$ $$H(a,b) = H(a) + H(b\mid a) \tag{2}$$ $$H(a,c) = H(a) + H(c\mid a) \tag{3}$$

Hence $$2H(a,b,c) = 2 H(a) + 2H(b\mid c,a)=\\=H(a,b)+H(a,c) + H(b,c\mid a) + H(b,c\mid a)- H(b|a)-H(c\mid a) \tag{4}$$

Then we need to show that

$$ H(b,c\mid a)- H(b\mid a)-H(c\mid a) \le 0$$

or

$$ H(b,c\mid a) \le H(b\mid a)+H(c\mid a) \tag{5}$$

But this is true, because it's the conditioned version of

$$H(X,Y) \le H(X) + H(Y)$$