For $A,B,C$ independent and normal, what is $I(A+B;\ A+C)$?

59 Views Asked by At

Say $A,B,C$ are mutually independent and normally distributed with zero mean but possibly different variances $\sigma_1,\sigma_2,\sigma_3$. What is the mutual information between $A+B$ and $A+C$? All the transformations I do to this problem land me back where I started, and it is not a nice integral to evaluate directly on a whim.

Equivalent to finding this is finding $H(A+B|A+C)$. This should be a routine problem, I just can't find the reference for it.

1

There are 1 best solutions below

0
On BEST ANSWER

The mutual information of two normal variables with correlation factor $\rho$ is

$$ I(X;Y) = -\frac{1}{2} \log(1-\rho^2) \tag{1}$$

See proof in any Information Theory textbook - eg.

In our case, letting $X=A+B$ and $Y=A+C$, we have

$\sigma_X^2 = \sigma_A^2+\sigma_B^2$, $\sigma_Y^2 = \sigma_A^2+\sigma_C^2$, $\sigma_{XY}=E(X Y) = \sigma_A^2$. Hence

$$ \rho = \frac{\sigma_A^2}{\sqrt{(\sigma_A^2+\sigma_B^2)(\sigma_A^2+\sigma_C^2)}}$$

You then replace in $(1)$ and you're done.