Informally:
Under what conditions can a conditional distribution (conditional density function/conditional mass function) be extended to a full joint distribution that's exchangeable? Intuitively, let's say that I fix some conditional distribution $\mathbb P[X_n=x_n | X_1=x_1, \dots, X_{n-1}=x_{n-1}]$. Under what conditions on this conditional distribution can I then find some joint distribution on $X_1, \dots, X_{n-1}$ such that the resulting joint distribution on $X_1, \dots, X_n$ is exchangeable, i.e. $\mathbb{P}[X_1=x_1, \dots, X_n=x_n]=\mathbb{P}[X_1=x_{\pi(1)}, \dots, X_n=x_{\pi(n)}]$ for any permutation $\pi$.
More formally, for discrete random variables:
Take a sample space $\mathcal X$ and a "conditional probability mass function" $r_n: \mathcal X \times \mathcal X^{n - 1}\to [0,1]$ that maps a sequence of points $(x_1, \dots, x_{n-1})\in\mathcal{X}^{n-1}$ to a probability mass function $r_n(\cdot|x_1, \dots, x_{n-1})$.
Under what conditions will there exist a probability mass function $p_{n-1}:\mathcal X^{n-1}\to[0,1]$ such that the joint distribution $p_n: \mathcal X^n\to[0,1]$ obtained by taking the product of $p_{n-1}$ and $r_n$: $$ p_n(x_1, \dots, x_n)=r_n(x_n|x_1, \dots, x_{n-1}) p_{n-1}(x_1,\dots, x_{n-1}) $$ is exchangeable, meaning that for any permutation of $n$ elements $\pi: [n] \to [n]$.
$$ p_n(x_1, \dots, x_n)=p_n(x_{\pi(1)}, \dots, x_{\pi(n)}) $$
Under what conditions might the extension $p_{n-1}$ be unique?
More formally, for general probability spaces:
Given a measurable space $(\mathcal X, \mathcal F)$, let's say that I have a "prediction rule at time-step $n$" $r_n:\mathcal F \times \mathcal X^{n-1} \to [0, 1]$. The prediction rule maps a sequence $(x_1, \dots, x_{n-1})\in \mathcal X^{n-1}$ to a probability measure $r_n(\cdot|x_1, \dots, x_{n-1})$ on $(\mathcal X, \mathcal F)$. More concretely, it satisfies:
- For every fixed $(x_1, \dots, x_{n-1})\in \mathcal X^{n-1}$, the function $A \mapsto r_n(A|x_1, \dots, x_{n-1})$ is a probability measure on $(\mathcal X, \mathcal F)$
- For every fixed $A\in \mathcal F$, the function $x_1, \dots, x_{n-1} \mapsto r_n(A|x_1, \dots, x_{n-1})$ is $\otimes_{i=1}^{n-1} \mathcal F$-measurable (with $\otimes_{i=1}^{n-1} \mathcal F$ denoting the product $\sigma$-algebra on $\mathcal X^{n-1}$)
In other words, $r_n$ is a Markov Kernel. These conditions simply ensure that given a probability measure $p_{n-1}$ on $(\mathcal X^{n-1}, \otimes_{i=1}^{n-1} \mathcal F)$ and $r_n$ we can uniqely extend them to a probability measure on $(\mathcal X^{n}, \otimes_{i=1}^{n} \mathcal F)$ in a natural way.
Under what conditions on $r_n$ will there exist a probability measure $p_{n-1}$ on $(\mathcal X^{n-1}, \otimes_{i=1}^{n-1} \mathcal F)$ (with $\otimes_{i=1}^{n-1} \mathcal F$ denoting the product $\sigma$-algebra) such that, for the resulting "semidirect product" measure $p_n$ that measure is exchangeable, i.e.: $$ p_n(A_1\times\dots\times A_n)=p(A_{\pi(1)}\times\dots\times A_{\pi(n)}) $$ for all permutations of $n$ elements $\pi: [n] \to [n]$.
Here, the semidirect product measure $p_n$ is defined as the unique measure satisfying: $$ p_n(A_1\times \dots \times A_n)=\int_{A_1\times \dots\times A_{n-1}} r(A_n|x^{(n-1)})p_{n-1}(dx^{(n-1)}) $$ with $x^{(n-1)}$ denoting a sequence element of $\mathcal X^{n-1}$.
Under what conditions will such a measure $p_{n-1}$ be unique?
Two notes:
- An intuitive idea might be that a sufficient condition is that $x_1, \dots, x_{n-1} \mapsto r_n(A| x_1, \dots, x_{n-1})$ be permutation invariant. However, one can come up with a simple counter-example on a finite space for $n=2$ showing that this is not the case.
- Is it even a necessary condition that $p_{n-1}$ must itself be exchangeable?