prove ordisprove on information theory

74 Views Asked by At

How does one prove or disprove these equations:

$$I(X+Y;Y|X)=0$$ and $$H(X|Z)-H(X|Y,Z)=H(Y|Z)-H(Y|X,Z)$$

and $$I((X_1,...X_n);(Y_1,\dots Y_n))=\sum_{i=1}^n H(Y_i|Y_1\dots Y_{i-1})-H(Y_i,X_i\dots X_n|Y_1\dots Y_{i-1},X_1\dots X_{i-1})+(H(X_i\dots X_n)|Y_1\dots Y_{i-1},X_1\dots X_{i-1})$$

Thank you.

for the fist question i trying this but i don't know how to get further let,s assume $$Z=X+Y$$ $$=> I(Z;Y|X)=H(Z|X)-H(Z|Y,X)=H(Y|X)-H(Z|Y)-H(X|Z,Y)$$

1

There are 1 best solutions below

1
On

Your first step is a good start. Now, when you define $Z=X+Y$ and get $$I(Z;Y|X)=H(Z|X)-H(Z|Y,X)$$

you should notice that $H(Z|Y,X)=0$ because knowing $Y$ and $X$ you know $Z$ (we could prove this in more detail if we wish)

Further, $H(Z | X) = H(X+Y | X) = H(Y|X)$ (again, this is intuitive but we could prove it). Then

$$I(X+Y;Y|X)=H(Y|X)$$ This is zero if and only iff $X=g(Y)$ ($X$ is a function of $Y$, knowing $X$ we can know $Y$). Hence the equality is in general not true.

The second one should be easier. See that $Z$ is conditioning everywhere. And prove that they are two differnt forms of writing $I(X;Y|Z)$

The third is more difficult. Let $X_{[i,n]}=(X_i, X_2 \cdots X_n)$ and $X_{[n]}=X_{[1,n]}=(X_1, X_2 \cdots X_n)$. We notice first that the summation correspond to $H(Y_{[n]})$ (chain rule, $n$ times), then write

$$I(X_{[n]}; Y_{[n]}) =H(Y_{[n]})-H(Y_{[n]}| X_{[n]})$$

Let's go for that second term.

$$H(Y_{[n]}| X_{[n]})=\sum H(Y_i | Y_{[1,i-1]}, X_{[1,n]})$$

We divide $ X_{[1,n]} = (X_{[1,i-1]} , X_{[i,n]})$ and ty to get $X_{[i,n]}$ "up" somehow. We recall $H(A|B)=H(B|A)+H(A)-H(B)$, hence... (can you go on from here?)