Can dependent variables have zero mutual information?

107 Views Asked by At

Let $X,Y$ be random variables. Assume that $X \sim N(0,1)$, and $X|Y \sim N(y,1)$.

Is this possible? What seems counter-intuitive to me is that this has the following properties:

  1. Clearly, knowing $Y$ is relevant for $X$: The distribution of $X$ depends on $Y$. i.e. $p_{X|Y}\neq p_X$.

  2. But nevertheless, the mutual information $I(X;Y)$ is zero.

If we extend the density functions of $X$ and $Y$ we get:

$$p_X=\int_{-\infty}^\infty p_{X|Y}\cdot p_Ydy$$

So we get

$$c_1\cdot e^{-\frac {x^2} 2}=\int_{-\infty}^\infty c_2\cdot e^{-\frac {(x-y)^2} 2}\cdot p_Ydy=c_1\cdot e^{-\frac {x^2} 2}\cdot\int_{-\infty}^\infty \frac {c_2}{c_1}\cdot e^{-\frac {y^2-2xy} 2}\cdot p_Ydy$$

So for this to work, we need a marginal density $p_Y$ such that

$$\int_{-\infty}^\infty \frac {c_2}{c_1}\cdot e^{-\frac {y^2-2xy} 2}\cdot p_Ydy=1\quad\quad \text{for all }x\in \mathbb R$$

This seems impossible.

But is there a way to show abstractly that this is impossible? i.e. without going into the specifics of the normal distribution. (e.g. just on the basis of properties $1$ and $2$)