Given a Markov chain $X \rightarrow Y \rightarrow Z$, under what condition $I(X;Y) = I(X;Z)$

Question

A theorem (The Data Processing Inequality) states that

if $X \rightarrow Y \rightarrow Z$, then $I(X ; Y ) \geq I ( X ; Z )$

Question: I was wondering under what conditions $I(X;Y) = I(X;Z)$?

---

The proof:

Using chain rule of mutual information, we have \begin{align*} I ( X ; Y , Z ) &= I ( X ; Y ) + I ( X ; Z | Y )\\ &= I ( X ; Z ) + I ( X ; Y | Z ) \end{align*} rewrite the above equalities, we have \begin{align*} I ( X ; Y ) + I ( X ; Z | Y ) &= I ( X ; Z ) + I ( X ; Y | Z )\\ I ( X ; Y ) &= I ( X ; Z ) + I ( X ; Y | Z )\\ I ( X ; Y ) &≥ I ( X ; Z ) \end{align*} To obtain $I(X;Y) = I(X;Z)$ indicates $I(X;Y|Z)=0$,

\begin{align*} I(X;Y|Z) &= 0\\ D_{\mathrm{KL}}[p(X,Y,Z) \| p(X|Z) p(Y|Z) p(Z)] &= 0\\ p(X,Y,Z) &= p(X|Z) p(Y|Z) p(Z) \end{align*} Would it be possible to further simplify it for the conditions?

Answer 1

As stated in Cover and Thomas's Elements of Information theory 2e (in the discussion of Theorem 2.8.1, the data processing inequality), you have equality iff $X \to Z \to Y$ is a Markov chain (think of why this is equivalent to $I(X;Y|Z) =0$ and the joint distribution of $X,Y$ given $Z$ under the markovian assumption I've stated and conditional independence).