Will the maximum entropy joint distribution given a known set of marginal distributions have the maximum plausible support?

103 Views Asked by At

Define $[n] = \{1, 2, \cdots, n\}$. Given a distribution $P: \{0, 1\}^{[n]} \to [0, 1]$ and a subset $S \subseteq [n]$, we can define the $S$-marginal of $P$, $P_S: \{0, 1\}^S \to [0, 1]$ as $$P_S(x) = \sum_{y \in \{0, 1\}^{[n] - S}} P(x \cup y).$$

Given a set of sets $\Sigma \subseteq 2^{[n]}$ indexing probability distributions $\phi_S : \{0, 1\}^S \to [0, 1]$, we can define $$P = \mathop{\operatorname{arg\,max}}\limits_{\substack{P:\ \forall S \in \Sigma,\ P_S = \phi_S\\\sum\limits_{x \in \{0, 1\}^{[n]}} P(x) = 1}} H(P)$$ where $H(P)$ is the Shannon entropy of $P$.

In other words, we can define $P$ as the maximum entropy distribution given the marginals $\{ \phi_S \mid S \in \Sigma \}$. Define $$\operatorname{supp}(P) = \{ x \mid x \in \{0, 1\}^{[n]}, P(x) > 0 \}.$$

You will notice each marginal constraints the support to a set: $$\operatorname{supp}_n(S) = \{ x \cup y \mid x \in \{0, 1\}^S, \phi_S(x) > 0, y \in \{0, 1\}^{[n] - S}\}$$ And it is clear that $\operatorname{supp}(P) \subseteq \bigcap\limits_{S \in \Sigma} \operatorname{supp}_n(S)$.

My question is if $\operatorname{supp}(P) = \bigcap\limits_{S \in \Sigma} \operatorname{supp}_n(S)$ when $P$ is well-defined.