Bounds on non-Gaussian channels with additive noise

32 Views Asked by At

I am trying to show that for independent random variables $X,Z$ and gaussian independent random variables $X^*,Z^*$, where $Var(X)=Var(X^*)=P$ and $Var(Z)=Var(Z^*)=N$ we have the inequality. $$ I(X;X+Z^*) \leq I(X^*;X^*+Z^*)\leq I(X^*;X^*+Z) $$

I can show the first inequality as $$ \begin{align} I(X;X+Z^*) &= h(X+Z^*) + h(X+Z^*|X)\\ &= h(X+Z^*) + h(Z^*)\\ &\leq h(X^*+Z^*) + h(Z^*)\\ &= I(X^*;X^*+Z^*) \end{align} $$ Where the inequality is because the gaussian is the maximum entropy distribution with a covariance constraint. And because $X\perp Z^*$, $Var(X+Z^*)=P+N$ which is constant, so the maximizing distribution $\tilde X+Z^*$ is gaussian implying $\tilde X = X^*$.

But I'm having trouble proving the second half of the inequality.

1

There are 1 best solutions below

1
On

Write

$$ I(X^*;X^*+Z)=h(X^*)-h(X^*|X^*+Z) $$

and note that the entropy $h(X^*|X^*+Z)$ is maximized when $X^*$ conditioned on $X^*+Z$ is Gaussian, which can be the case when $X^*$ and $X^*+Z$ are jointly Gaussian, which, in turn, can be the case when $Z$ is Gaussian.