Entropy of extractions

Question

A box contains $3$ white and $6$ black balls. We draw $2$ balls consequentially without replacement. Find the entropy of first and second extractions and the entropy for both of them.

Answer 1

For the first case, when $2$ balls are drawn sequentially, the corresponding pmf is

$$P(B_1B_2)=\frac{6}{9}.\frac{5}{8}=\frac{5}{12}$$

$$P(W_1B_2)=\frac{3}{9}.\frac{6}{8}=\frac{1}{4}$$

$$P(B_1W_2)=\frac{6}{9}.\frac{3}{8}=\frac{1}{4}$$

$$P(W_1W_2)=\frac{3}{9}.\frac{2}{8}=\frac{1}{12}$$

Hence, the entropy is $$H(P) = -\sum\limits_{i}p_{i}\log_2{p_i}=1.825011$$

For the second case, when $2$ balls are drawn sequentially, the corresponding pmf is

$$P(2B) = \frac{{6 \choose 2}}{{9 \choose 2}} = \frac{5}{12}$$

$$P(2W) = \frac{{3 \choose 2}}{{9 \choose 2}} = \frac{1}{12}$$

$$P(1B1W) = \frac{{3 \choose 1}{6 \choose 1}}{{9 \choose 2}} = \frac{1}{2}$$

Hence, the entropy is $$H(P) = -\sum\limits_{i}p_{i}\log_2{p_i}=1.325011$$

Which is as per our intuition that in the second case entropy is lower because the uncertainty is lower (the order of drawing need not be considered).