KL Divergence of (y|z) in terms of KL Divergence of (x|y) and (x|z)

Question

Consider there are three distributions $X$, $Y$ and $Z$ and all three have same support. Then is there some relationship of the following form which holds? \begin{align} D_{KL}(Y||Z) \stackrel{?}{=} f(D_{KL}(X||Y) , D_{KL}(X||Z)) \end{align} where $f$ is some arbitrary function. If yes, please do point out an appropriate $f$ and any link to literature will be appreciated.

Answer 1

I claim that there is no such function, which I show by contradiction.

Suppose that such a function $f$ exists.

Let $X,Y,Z$ be rvs on $\{0, 1\}$ such that $X$ is uniform and $Y=Z\sim\text{Bernoulli}(p)$. Then, $f(d, d) = D_{KL}(Y||Z)=0$, where $$ d= D_{KL}(X||Y)= -1 - \frac{1}{2}\log [p(1-p)]. $$ Note that an arbitrary nonnnegative $d$ can be achieved by for some $p$ strictly between $0$ and $1$. Hence, $f(d,d) = 0$ for all $d\ge 0$.

Now, let $X,Y,Z$ be rvs on $\{0, 1\}$ such that $X$ is uniform, $Y\sim \text{Bernoulli}(p)$ and $Z\sim\text{Bernoulli}(1-p)$ for $p$ strictly between $0$ and $1/2$. Then, $$ D_{KL}(Y||Z)=p \log \frac{p}{1-p} + (1-p)\log \frac{1-p}{p} = (2p - 1)\log \frac{p}{1-p}, $$ which is not zero since $p$ is strictly less than $1/2$.

However, $$ D_{KL}(X||Y) = D_{KL}(X||Z) = -1 - \frac{1}{2}\log \frac{p}{1-p}. $$ This implies that $0 \ne D_{KL}(Y||Z)=f(d,d)$ where $d = D_{KL}(X||Y)\ge 0$, and we have a contradiction.