Find $H(Y,Z)$ and $I(X,Y)$ given $p(x,y,z)$

Question

Given the joint probability distribution function: $$p(x, y, z)=p(x)p(y|x)p(z|x)$$ with:

$ X: p(X=0)= p(X=1)= 1/4, \ p(X=2)=1/2 ,$ $ Y: p(Y=0| X=0)= p(Y=1| X=1)= 1, \ p(Y=0| X=2) = p(Y=1|X=2)=1/2 ,$ $ Z: p(Z=0|X=0)= p(Z=0|X=1)=p(Z=1|X=2)=1:$

• Question 1:

Is it right that $H(Z|Y)=H(Y│Z)=0$? I am asking because I want to find $H(Y,Z)$ using: $$ H(Y,Z)=H(Y)+H(Z|Y)=H(Z)+H(Y│Z). $$

• Question 2: Is it right that I can find $I(X,Y)$ in this way?: \begin{split} I(X,Y)&=Η(Y)-H(Y│X) \\ &=1-\sum_{x\in X }\sum_{y\in Y}p(Y=y|X=x)\log_2(p(Y=y|X=x))\\ &=1-(1 \log_2⁡(1)+0+1/2 \log_2(1⁡/2)+0+1 log_2⁡(1)+ 1/2 \log_2(⁡1/2))\\ &=2.\end{split}

Thank you very much!

Answer 1

Note that if $H(Y|Z)=0$ then $Y$ should be a deterministic function of $Z$ which is not the case here obviously. But indeed it can be shown that $Z$ and $Y$ are independent and hence $H(Z|Y)=H(Z)$ and $H(Y|Z)=H(Y)$. To see the Independence, we verify in directly. Note that $p(Y=y)$ is given $$ p(Y=0)=\frac 14+\frac 12\frac 12=\frac 12\implies p(Y=1)= \frac 12\\ p(Z=0)=\frac 14+\frac 14=\frac 12\implies p(Z=1)=\frac 12. $$ But since $p(X=0|Z=0)=\frac{1\times p(X=0)}{p(Z=0)}=\frac 12$, $p(X=1|Z=0)=\frac{1\times p(X=1)}{p(Z=0)}=\frac 12$, $p(X=2|Z=1)=1$: $$ p(Y=0|Z=0)=\sum_i p(X=i|Z=0)p(Y=0|X=i)=\frac 12\times 1+\frac 12\times 0=\frac 12\\ p(Y=1|Z=0)=\sum_i p(X=i|Z=0)p(Y=1|X=i)=\frac 12\times 0+\frac 12\times 1=\frac 12\\ p(Y=0|Z=1)=\sum_i p(X=i|Z=1)p(Y=0|X=i)=1\times \frac 12=\frac 12, p(Y=1|Z=1)=\sum_i p(X=i|Z=1)p(Y=1|X=i)=1\times \frac 12=\frac 12. $$ So $Y$ and $Z$ are independent.

Question 2: note that $$ H(Y|X)=-\sum_{x\in X }\sum_{y\in Y}p(Y=y,X=x)\log_2(p(Y=y|X=x)) $$ So what you write there is wrong both the expression and its sign. See that (why?): $$ H(Y|X)=p(X=2)H(Y|X=2)=\frac 12\log_2 2=\frac 12. $$ Hence: $I(X;Y)=1-\frac 12=\frac 12$.