Probability distribution and entropy of a set of possible outcomes

99 Views Asked by At

Suppose $X$ and $Y$ are discrete random variables and that the possible outcomes of $(X, Y)$ are $(1, 2), (1, 3), (0, 0), (0, 1), (0, 2), (0, 3), (2, 3)$ and that each of these is equally probable. Calculate $H(X), H(Y)$ and $H(X, Y)$, and hence determine whether $X$ and $Y$ are independent.

I know $X$ and $Y$ are independent if $H(X)+ H(Y)= H(X, Y)$

I wanted to know if I have done this correctly as I am unsure with some of the probability distributions.

I get $H(X)=-\left(\dfrac{2}{7}\log_2 \left(\dfrac{2}{7}\right)+\dfrac{1}{7}\log_2 \left(\dfrac{1}{7}\right)+\dfrac{4}{7}\log_2 \left(\dfrac{4}{7}\right)\right)$ $=1.379$

$H(Y)=-\left(\dfrac{2}{7}\log_2 \left(\dfrac{1}{7}\right)+\dfrac{2}{7}\log_2 \left(\dfrac{2}{7}\right)+\dfrac{3}{7}\log_2 \left(\dfrac{3}{7}\right)\right)$ $=1.842$

and $H(X,Y) = -\dfrac{7}{7}\log_2 \left(\dfrac{1}{7}\right)=2.807$

So hence $X$ and $Y$ are not independant. Are these calculations correct for the probability distributions? I am particulary doubtful for $H(X,Y)$