Is Entropy of a Discrete Random Variable Condition on a Continuous Random Variable Always bigger than or Equal to Zero?

Question

Let $X$ be a discrete random variable and $Y$ be a continuous random variable. Is the conditional entropy of $X$ given $Y$ always positive, i.e., $H(X|Y)\ge0$?

Answer 1

Kind of trivial. If you know that the Shannon entropy of a discrete variable is non-negative, and you accept the natural extension of the definition of (discrete) conditional entropy, when the conditioning variable is continuous with a density function, you'd write

$$H(X|Y) = \int f_Y(y) H(X|Y=y) \, dy$$

Now, $H(X|Y=y) \ge 0 $ because that's the entropy of a discrete variable (with pmf $P(X|Y=y)$). Then $H(X|Y) \ge 0$.