Does the quantity $D_{KL}(P_{(X,Y)} | Q_{(X,Y)}) - D_{KL}(P_X | Q_X)$ have a name?

95 Views Asked by At

I'm interested in the information gained from using a joined distribution, compared to just one of it's marginals, that is:

$$\begin{align}g_x &= D_{KL}(P_{(X,Y)} | Q_{(X,Y)}) - D_{KL}(P_X | Q_X) \\\quad \text{and} \quad g_y &= D_{KL}(P_{(X,Y)} | Q_{(X,Y)}) - D_{KL}(P_Y | Q_Y)\end{align}$$

where $P$ and $Q$ are distributions.

Clearly we have $$\max\{D_{KL}(P_X | Q_X), D_{KL}(P_Y | Q_Y)\} \le D_{KL}(P_{(X,Y)} | Q_{(X,Y)}) \le D_{KL}(P_X | Q_X) + D_{KL}(P_Y | Q_Y)$$ with equality when $X$ and $Y$ are independent, and so $0 \le g_x \le D_{KL}(P_Y | Q_Y)$ and $0 \le g_y \le D_{KL}(P_X | Q_X)$.

Superficially they seem similar to to the uncertainty coefficients: $$ C_{XY} = \frac{\operatorname{I}(X;Y)}{H(Y)} ~~~~\mbox{and}~~~~ C_{YX} = \frac{\operatorname{I}(X;Y)}{H(X)}. $$

I'm wondering if these have been studied before, and/or have a name I may refer them by?