How to show the following relation is true in information theory?

68 Views Asked by At

Suppose $X_1,X_2$ are two random variables which can take values from the set $\mathcal{X}$ with uniform distribution. Further, $N$ is a Guassian random variable with zero mean and unit variance. In this case how to show that the following inequality is true $$H(X_1|X_1+X_2+N)\geq H(X_1|X_1+X_2).$$ Any help in this regard will be much appreciated. Thanks in advance.

1

There are 1 best solutions below

0
On

This is an application of data processing inequality applied to the Markov chain: $$ X_1-\!\!\!\!\ominus\!\!\!\!- X_1+X_2 -\!\!\!\!\ominus\!\!\!\!-X_1+X_2+N $$ To see this, note that a simple application of conditioning inequality for entropy implies that: $$ H(X_1|X_1+X_2+N)\geq H(X_1|X_1+X_2+N,X_1+X_2) $$ But $H(X_1|X_1+X_2+N,X_1+X_2)=H(X_1|X_1+X_2)$.