Entropy and the probability to guess

216 Views Asked by At

Let $X$ be a discrete random variable and suppose that we choose a random value $X=x_1$. Let $A$ be an event such that $H[ X \mid A] = k$, where $$H[X \mid A] = - \sum_{x} P[X =x \mid A] \log_2( P[X =x \mid A]).$$

Is it correct to interpret the entropy bound by saying that, after having observed $A$, the probability of guessing $x_1$ is at most $2^{-k}$? This seems to be true when $X$ is uniformly distributed, but is there such a lower bound in general?