I'm interested in the problem "find the closest probability measure in $KL$-divergence of a set of measures $Q$ to a given measure $p$."
Specifically take $Q$ to be the set of product measures, that is measures where each component is independent: $$Q = \big\{ q(x): q(x) = \prod_{i=1}^d q_i(x_i) : x\in \mathbb{R}^d\big\}.$$
Suppose $p$ is in the exponential family.
How do I find $\arg\min_{q\in Q} KL(q||p)$ for given $p$?
- Is this the same as the projection of $p$ onto $Q$ in some metric?
- Is it possible to just take the marginals $p(x_i)$ and combine them -- getting a product distribution; would this be the $KL$-minimizer? If not, what would this be?