Conditional Entropy Inequality $H(X) \geq H(X | Y = y)$

Question

Let $H(X)$ denote the entropy of random variable $X$. It is well known that $H(X) \geq H(X|Y)$ where $X$ and $Y$ are two random variables.

Is it also true that $H(X) \geq H(X | Y = y)$ for any value of $Y=y$?

Answer 1

No. Imagine if $X$ is a mixture of two random variables, one with high entropy and one with low, then the actual entropy of $X$ will be in between the two mixture components. But if we condition upon the event that $X$ is equal to the high entropy mixture component, then that will have higher entropy.

Example: Let $Y$ be the outcome of a coin flip. $\mathbb{P}(Y=H)=.99$ and $\mathbb{P}(Y=T)=.01$. If $Y$ is heads, then $X=0$. If $Y$ is tails, then $X$ is either $-1$ or $1$, this time based on the flip of a fair coin.

Then $\mathbb{P}(X=0)=.99$, $\mathbb{P}(X=-1)=\mathbb{P}(X=1)=.005$. The entropy is roughly $.063$. But if we condition on $Y=T$, then $X|Y=T$ is a $-1$ or $1$ each with probability $.5$, which has entropy around $.69$.