Weakened version of Fano's inequality

106 Views Asked by At

How does the weakened version of the Fano's inequality $1+P_{e}\log(|X|-1) \geq H(X|Y)$ make sense because trivially $H(X|Y) \leq 1$. What am I missing?

1

There are 1 best solutions below

0
On

The standard formulation is $$ H(X | Y) \le H({1}\{X \ne Y\}) + P(X \ne Y) \log( |\mathcal{X}| - 1) $$ but $1\{X \ne Y\}$ is just a Bernoulli($p$) random variable for some $p$. The entropy of this RV is maximised by $p= 1/2$ and at this value the entropy is $H(\text{Bernoulli}(1/2)= 1$.