Csiszar Sum Identity

402 Views Asked by At

Let the random variables $M \to X^n \to (Y^n,Z^n) $ form a markov chain.

$M \in [1:2^{nR}]$, $\hspace{2pt}$ $X^n, Y^n, Z^n \in \{0,1\}^{n}$, where $n \in \mathbb{R}$ and $R \in (0,1)$ are fixed numbers. I am actually describing the problem setting for discrete memoryless wiretap channel as given in the book by EL Gamal. In the converse proof of the theorem for secrecy capacity, I can not understand the equality of these 3 particular lines.

$$ nR \leq \sum_{i=0}^{n}(I(M;Y_i|Y^{i-1})-I(M;Z_i|Z^n_{i+1})+n(\epsilon + \epsilon_n))$$ $$ = \sum_{i=0}^{n}(I(M,Z^n_{i+1};Y_i|Y^{i-1})-I(M,Y^{i-1};Z_i|Z^n_{i+1})+n(\epsilon + \epsilon_n))$$ $$ = \sum_{i=0}^{n}(I(M;Y_i|Y^{i-1},Z^n_{i+1})-I(M;Z_i|Z^n_{i+1},Y^{i-1})+n(\epsilon + \epsilon_n))$$ I can't understand how 1st and 2nd eXpressions' RHS are equal, nor can I understand how 2nd and 3rd expressions' RHS are same. The justification for both the steps is given as Csiszar Sum Indentity.

Note - $X_a^b$ means the elements of the vector $X_a$ to $X_b$, both inclusive. $X^b$ means $X_1$ to $X_b$.

1

There are 1 best solutions below

2
On BEST ANSWER

Let us recall the Csiszar sum identity.

For any random variables $X$ and sequences of random variables $Y_1^n, Z_1^n$ it holds that $$ \sum_i I(Y^{i-1};Z_i|Z_{i+1}^n, X) = \sum_i I(Z_{i+1}^n;Y_i|Y^{i-1}, X)$$

For the first equality, notice that $$ I(M, Z_{i+1}^n; Y_i|Y^{i-1}) = I(M;Y_i|Y^{i-1}) + I(Z_{i+1}^n; Y_i|Y^{i-1}, M).$$

Similarly, $$ I(M, Y^{i-1}; Z_i|Z_{i+1}^n) = I(M;Z_i|Z_{i+1}^n) + I(Y^{i-1}; Z_i|Z_{i+1}^n, M).$$

So, to show the first equality, it suffices to show that $$ \sum_i I(Y^{i-1};Z_i|Z_{i+1}^n, M) = \sum_i I(Z_{i+1}^n;Y_i|Y^{i-1}, M).$$ But this is precisely the Csiszar sum identity (applied with $X= M$).

For the second equality, we can expand mutual information in the second way to see that $$ I(M, Z_{i+1}^n; Y_i|Y^{i-1}) = I(Z_{i+1}^n;Y_i|Y^{i-1}) + I(M; Y_i|Y^{i-1}, Z_{i+1}^n)\\ I(M, Y^{i-1}; Z_i|Z_{i+1}^n) = I(Y^{i-1};Z_i|Z_{i+1}^n) + I(M; Z_i|Z_{i+1}^n, Y^{i-1}).$$

Again, the equality will follow if $$ \sum_i I(Z_{i+1}^n; Y_i|Y^{i-1}) = \sum_i I(Y^{i-1}; Z_i|Z_{i+1}^n).$$

But this is again just the Csiszar sum identity (applied with the trivial random variable $X \equiv 1$)