Non-negativity of interaction information for special trivariate case

99 Views Asked by At

Consider a discrete trivariate distribution $P(X_1, X_2, Y)$, which satisfies $$ p(x_1, x_2, y) = \min( p(x_1,y), p(x_2,y) ), $$ for all $x_1$ and $x_2$ for which $p(x_1, x_2) > 0$ and for all values of $y$ (including those at which $p(x_1, x_2, y) = 0$). I am trying to show that then the interaction-information is non-negative $$ I(X_1;Y) + I(X_2;Y) - I(X_1 X_2;Y) \ge 0 $$

I need either a rigorous proof or, alternatively, a counterexample that disproves the theorem.

Addendum 1: I use the shorthand notation $p(x_1, x_2, y) := P(X_1=x_1, X_2 = x_2, Y=y)$

Addendum 2: for a similar inequality and its proof see here. This might provide some inspiration. For theoretical references on non-Shannon inequalities see here and here.

Addendum 3: Note that the canonical example of negative interaction-information, the XOR gate, does not satisfy the theorem hypothesis. For the XOR gate $p(x_1, x_2, y) = \min \left( \ p(x_1,y), \ p(x_2,y) \ \right)$ holds for all $x_1, x_2, y$ such that $p(x_1, x_2, y)>0$ but not for all $y$ once we pick $x_1, x_2$ points for which $p(x_1, x_2)>0$, i.e. also the $y$s for which $p(x_1, x_2, y)=0$.