Calculating Entropy and Information Gain of a Variable

Question

I have the following values for two random variables. I need to compute the following values:

a. H(Y) b. H(Y|X) c. and finally IG(Y|X)

I will show what I have calculated so far.

a. H(Y) = -(.5*log(.5) + .5*log(.5)) = 0.301

b. H(Y|X) = .5? I know you calculate it based off given a yes or know the entropy. I tried thinking about it as -(.5*log(.5) + .5log(.5)), but I feel like this is wrong. Any assistance or guidance would be greatly appreciated for this one.

c. IG(Y|X) = H(Y) - H(Y|X) = 0.301 - .5 Again, I know these values need to be subtracted, but I'm still confused about calculating H(Y|X).

Thank you everyone very much.

Answer 1

$$\mathsf H(Y\mid X) = - \sum_x p_X(x) \sum_y p_{Y\mid X}(y\mid x)\log p_{Y\mid X}(y\mid x)$$

or

$$\mathsf H(Y\mid X) ~=~ \sum_{x,y} p_{X,Y}(x,y)\log \frac{p_{X}(x)}{p_{X,Y}(x,y)}$$

Hint: (the latter will be a sum of six terms.)