Mutual Information

76 Views Asked by At

Given $X_1$ and $X_2$ are independent, we have

\begin{align} I(X_1,X_2;Y_1,Y_2) & = I(X_1;Y_1,Y_2) + I(X_2;Y_1,Y_2\mid X_1) \\[1ex] & = I(X_1;Y_1) + I(X_1;Y_2\mid Y_1) + I(X_2;Y_2\mid X_1) + I(X_2;Y_1\mid X_1,Y_2) \\[1ex] & \ge I(X_1;Y_1) + I(X_2;Y_2\mid X_1) \\[1ex] & = I(X_1;Y_1) + I(X_1,X_2;Y_2) \tag a \\[1ex] & \ge I(X_1;Y_1) + I(X_2;Y_2) \end{align}

where equation (a) in the line before last follows by the independence of $X_1$ and $X_2$. I don't understand this (a) part.

I can write $I(X_1,X_2;Y_2) = I(X_1;Y_2) + I(X_2;Y_2\mid X_1)$, but how does $I(X_1;Y_2)$ have anything to do with the independence of $X_1$ and $X_2$?

1

There are 1 best solutions below

2
On

I think it is possible to obtain it in another way. From the third line, because $I(X_2;X_1,Y_2)=I(X_2;X_1)+I(X_2;Y_2|X_1)=I(X_2;Y_2|X_1)$ if $X_1$ and $X_2$ are independent, thus we have $I(X_2;Y_2|X_1)\ge I(X_2;Y_2)$, which establishes the conclusion.