Difference of KL Divergences

48 Views Asked by At

I am interested in the difference of KL divergences \begin{align} KL&(p(x)||q(x))-KL(p(x|y)||q(x|y)) = \\ &\int p(x)\log\dfrac{p(x)}{q(x)}dx-\int p(y)\int p(x|y)\log\dfrac{p(x|y)}{q(x|y)}dxdy \end{align} I want to make this difference equal to zero. I can't just minimize this quantity as the difference of KL's can be negative. $p(x|y)$ is intractable so I can not directly compute the conditional KL divergence term. Is there any way that I can choose a $q(x|y)$ to know what the conditional KL evaluates to without needing to compute the KL directly? Or is there some theorems/resources that explore properties of the differences of KL divergences that I can hopefully leverage?