I am trying to define rigorously the conditional distribution of random variables for myself.
Suppose I have a probability space $(\Omega,\Sigma,\mathbb P)$, a measurable random variable $X$ such that $\mathcal X=\text{supp }\{X\}$ and $\mathcal P(\mathcal X) = \{ \mu: (\mathcal X,\sigma(X),\mu)\text{ is a probability space} \}$ the support and the simplex of $X$.
In terms of motivation, for a measurable random variable $U$, I want $p_{X|U}$ to be a $U$-measurable random measure in $\mathcal P(\mathcal X)$ such that if we condition on the event $U=u$ for some $u$, then $p_{X|U}=p_{X|U=u}$ (the usual definition of conditional distribution I guess). This is non rigorous and here is an approach to do that :
Define, for $x\in\mathcal X$, $(\mathcal X,\sigma(X),\delta_x)$ such that for any $A\in\sigma(X)$, $\delta_x(A)=\mathbf 1(x\in A)$. We start by defining $p_{X|X}=\delta_X$ and then for a given measurable random variable $U$, we define $p_{X|U}=\mathbb E[p_{X|X}|U]$.
In order for this to work I have to define $\mathbb E[p_{X|X}|U]$ correctly. Since for any $A\in\sigma(X)$ both $p_{X|X}(A)$ and $p_{X|U}(A)$ are supposed to be measurable random variables, it feels we define $\mathbb E[p_{X|X}|U]$ to be the random measure such that for all $A\in\sigma(X)$, $p_{X|U}(A) = \mathbb E[p_{X|X}(A)|U]$. If we do that we satisfy the property that conditioned on $U=u$, we have $p_{X|U}=p_{X|U=u}$.
I think in order to finish my definition of $p_{X|U}$, I have to show that for any $\omega\in\Omega$, conditioned on $\omega$ we have that $(\mathcal P(\mathcal X),\mathcal F,p_{X|U})$ is a probability space. I would like to know if there is a way to define $\mathcal F$ as the Borel sigma algebra of some topological space on $\mathcal P(\mathcal X)$ independently of the random variable $U$ (so that I can use the same topology for all my random variables), this means that idealy $\mathcal F$ would solely depend on $\mathcal X$. I think this is a bit similar to defining all real random variable to be measurable functions that goes to the Borel sigma algebra on $\mathbb R$ with the usual metric topology.
Also if anything is not well defined, or unclear, I would appreciate any comments.
Here is a possible solution : \begin{align*} \mathcal F &= \{ B\subseteq \mathcal P(\mathcal X) : \forall A\in\sigma(X),\{\mu(A):\mu\in B\}\in \mathcal B([0,1]) \}\\ &=\bigcap_{A\in\sigma(X)} \{ B\subseteq \mathcal P(\mathcal X) : \{\mu(A):\mu\in B\}\in \mathcal B([0,1]) \} \end{align*} In this $\mathcal B([0,1])$ is the usual Borel sigma algebra on $[0,1]$.
I think it could work with that definition, but I am not sure on how to prove that for any $U$, $p_{X|U}$ is measurable unless it is sufficient to say that $p_{X|U}(A)$ is $[0,1]$-Borel measurable for all $A\in\sigma(X)$.
If this is correct, I am wondering if it is also possible to generate $\mathcal F$ using convex subsets of $\mathcal P(\mathcal X)$ so that $\mathcal F$ is the sigma algebra generated by all convex subsets of $\mathcal P(\mathcal X)$. Then maybe this can be reduced to the convex sets that are convex hulls of sets of measures with finite (or countable maybe ?) number of elements. This would have a great importance in my work (information theory) since we deal a lot of convex sets, convex functions and this kind of things. Any help on that would be much appreciated.