Calculate the Mutual Information between $X$ and $X^2$, where $X$ is uniformly distributed.

Question

Let us consider two random variables, $X$ and $Y$. Let $X$ be uniformly distributed in $[-1,1]$ and let $Y=X^{2}$.

Is it possible to calculate the Mutual Information between them? E what is the result? I tried to estimate it, but without success.

Answer 1

On their supports $X,\,Y$ have respective PDFs $\frac12,\,\frac12 y^{-1/2}$, while the joint PDF is $p(x,\,y):=\frac12\delta(x^2-y)$. The mutual information is $$\int_{-1}^1dx\int_0^1dy\frac12\delta(x^2-y)\ln (4\sqrt{y}p(x,\,y))\\=\int_0^1dx\int_0^1dy\delta(x^2-y)\ln (4\sqrt{y}p(x,\,y))\\=\int_0^1dx\ln (4xp(x,\,x^2))=\int_0^1dx\ln (2x\delta(0))=\infty.$$