Consider the events X = {Raining, Not raining}, Y = {Cloudy, Not cloudy} and the following probabilities.
| cloudy | not cloudy | |
|---|---|---|
| raining | 24/100 | 1/100 |
| not raining | 25/100 | 50/100 |
What is the entropy of cloudiness, given the knowledge of whether or not it is raining?
Here is my solution.
$$\begin{equation}\begin{aligned} H(Y|X) &= \sum_{x \in X} p(x) H(Y|X=x) \\ &= p(rain)H(cloudy|rain) + p(no rain)H(cloudy|no rain)\\ &=0.25 * H(cloudy|rain) + 0.75 * H(cloudy|no rain) \end{aligned}\end{equation}$$
According to my calculations H(cloudy|rain) is approximately 0.24 and H(cloudy|no rain) 0.53. Thus
$$\begin{equation}\begin{aligned} H(Y|X) &= 0.25 * 0.24 + 0.75 * 0.53 = 0.46 \end{aligned}\end{equation}$$
However the result is supposed to be 0.75. Can someone tell me where I am doing wrong? Thanks.
Your formula is false. $$H(Y|X) = \sum_{x \in X} p(x) H(Y|X = x)$$
$p(rain) = 0.25$ and $p(norain) = 0.75$ $$H(Y|X=rain) \\= -P(Y=cloudy|X=rain)log_2(P(Y=cloudy|X=rain)) - P(Y=notcloudy|X=rain)log_2(P(Y=notcloudy|X=rain)) \\= -(\frac{24}{25}log_2(24/25) +\frac{1}{25}log_2(1/25)) \\ = 0.242292...$$
$$H(Y|X=notraining) = -(\frac{25}{75}log_2(25/75) +\frac{50}{75}log_2(50/75)) = 0.918295...$$
Then $0.25 \star 0.242292... + 0.75 \star 0.918295... \sim 0.7492949...$