The interaction information (II) in 3D is defined as a generalization of the 2D mutual information (MI).
$$ \begin{eqnarray} I(X:Y:Z) &=& I(X:Y) - I(X:Y|Z) \\ &=& H(X) + H(Y)+H(Z) - H(XY) - H(XZ) - H(YZ) + H(XYZ) \end{eqnarray} $$
II can be negative, for example for $Z = \mathrm{XOR}(X, Y)$. It is easy to convince oneself of this fact by simply calculating all terms for the XOR example and confirming that it is indeed negative.
My question is about the geometric interpretation of this fact. The multivariate mutual information is typically illustrated by this three-way Venn diagram, where the entropies of each variable (which are positive quantities) are subdivided into shared and unique parts. Naively looking at this diagram it feels like each slice of a positive quantity should also be a positive quantity. How is it possible that the centre piece of this diagram is negative geometrically? Where does the naive intuition go wrong? Is there something better than a Venn diagram to geometrically illustrate this result?
