What property of integration over probability densities allows this?

Question

Here is the concerning part of the equation that I am trying to understand dealing with the variational lower bound and Kullback-Leibler divergence.

$$ \begin{aligned} &- \int_Z q(Z) \log \frac{p(Z|X)}{q(Z)} \\ &= -\Big( \int_Z q(Z) \log \frac{p(X,Z)}{q(Z)} - \int_Z q(Z) \log p(X) \Big) \end{aligned} $$

I just don't get what property of integration would give this. $p(X, Z)$ would be equal to $p(X|Z)p(Z)$ by bayes theorem, so it must be doing that somewhat in reverse, but I can't justify why that would be

Answer 1

$P(Z|X)=\frac{P(Z,X)}{P(X)}=\frac{P(X,Z)}{P(X)}$ since $P(X,Z)=P(Z,X)$.

Edit

By the above equality, you have $$\int_Z q(Z)\log\frac{p(Z|X)}{q(Z)}= \int_Z q(Z) \log \frac{p(X,Z)}{q(Z)p(X)} =\int_Z q(Z)\left[\log\frac{p(X,Z)}{q(Z)}-\log p(X)\right]$$

Answer 2

Since:

$$\frac{p(Z \mid X)p(X)}{q(Z)}= p(X \mid Z) = \frac{p(X, Z)}{q(Z)} $$

we have:

$$log \left ( \frac{p(Z \mid X)p(X)}{q(Z)} \right ) = log\left ( \frac{p(X, Z)}{q(Z)} \right )$$

from which, finally:

$$log\left ( \frac{p(X, Z)}{q(Z)} \right ) = log\left ( \frac{p(Z \mid X)}{q(Z)} \right ) + log(p(X))$$