Finding conditional entropy

69 Views Asked by At

Let $X,Y$ be independent R.V. such that $X,Y$~$Ber(0.5)$, and let $Z=X\bigotimes Y=X+Y(mod2)$

We're asked to find $H(Z|X)$, where $H$ is the entropy.

I've calculated that $H(X)=1$ yet I am unsure how to continue from there.

1

There are 1 best solutions below

0
On

If $X$ is i.i.d. $\mathrm{Bern}(p)$ and $Y$ is i.i.d. $\mathrm{Bern}(r)$ such that $X$ and $Y$ are independent, then $Z = X\oplus Y = X + Y\,\,\mbox{(mod $2$)}\hspace{6pt}$is $\mathrm{Bern}(p\ast r)$ where $p\ast r = p(1-r) + r(1-p)$. Then we have that $H(X) = H(p)$ and $H(Z) = H(p\ast r)$, and the joint entropy

$$H(X,Z) = H(X,Y) = H(X) + H(Y) = H(p) + H(r).$$

Thus, $H(Z|X) = H(r)$, because $H(X,Z) = H(X) + H(Z|X)$ (by the "chain rule").

As you say, $H(X) = H(1/2) = 1$...