Mutual information of $X,Y$ depending on $X$

104 Views Asked by At

Given $X,Y \in \{0,1\}$ and $P(X = 0) = s$, as well as: $$P(Y=0|X=0) = P(Y=1|X=1) = (1-t)$$ and $$P(Y=0|X=1) = P(Y=1|X=0) = t$$ I want to show that:

$$I(X,Y) = H(s + t - 2ts) - H(t)$$

My first approach is: $$I(X,Y) = H(Y) - H(Y|X)$$ Consider $$H(Y|X) = P(X=0) H(Y|X=0) + P(X=1)H(Y|X=1) = ... \\= -((1-t) \log_2(1-t) + t \log_2 t) = H(t)$$

However, how can I show that $H(Y) = H(s + t - 2ts)$? I tried to compute $P(Y=0)$ and $P(Y=1)$ but without success. Any hints?

2

There are 2 best solutions below

0
On BEST ANSWER

$$\begin{align} P(Y=0) =& \sum_x P(Y=0, X=x) \\ =& \sum_x P(Y=0 \mid X =x) P(X=x) \\ =& P(Y=0 \mid X =0) P(X=0) + P(Y=0 \mid X =1) P(X=1) \\ =& (1-t) s + t (1-s)\\ =& s+ t -2 st \end{align}$$

1
On

You'll need to use that $P(Y = 0) = E[P(Y = 0 | X)] = \sum_{i \in \{0,1\}} P(Y = 0 | X=i) P(X=i)$.