Maximization of information over set of non-injective functions

36 Views Asked by At

Let $X$, $Y$, $Z$ be discrete random variables, with $Y$ and $Z$ independent. Does the following equality hold if $Z$ is independent also of $X$? $$ \max_{f_{Y,Z}} \big\{ \ I(X; f_{Y,Z}(Y,Z)) \ \big\} = \max_{f_X, f_Y} \big \{ \ I(X; f_Y(Y), f_Z(Z)) \ \big \} $$ where the maximization is taken over all non-injective, deterministic functions.

P.S.: See this for the inequality version of the question.

1

There are 1 best solutions below

0
On BEST ANSWER

Take the classic example of $X,Y,Z$ binary, $Y, Z \sim \mathrm{Bern}(1/2)$ independently, and $X = Y \oplus Z$ - note that in this case $X$ is independent of $Y$ and independent of $Z$ (but not independent of $(Y,Z),$ of course). The only non-injective functions of one bit are constants, so the right hand side is $0$. On the left hand side, we can use the non-injective function $f_{Y,Z}(Y,Z) = Y \oplus Z$ to violate the desired equality.