Sorry for the poor title. Not sure how to ask this question without formalism. Let $X$ be a metric space and $\Delta(X)$ be the space of all probability measures, endowed with with topology of weak convergence (i.e., weak*). This is itself metrizable. So endow $X\times\Delta(X)$ with the product topology, and finally, let $\Delta(X\times \Delta(X))$ be the space of all probability measures thereover.
Now say that $F \in \Delta(X\times \Delta(X))$. For all $\mu \in supp[marg_{\Delta(X)} F]$, let $\mu^n$ be a sequence converging to $\mu$. Define $$\xi_n: X\times supp[marg_{\Delta(X)} F] \to X\times \Delta(X)$$
as the following map:
$$\xi_n: (x,\mu) \mapsto (x,\mu^n).$$
Finally, define $F^n \in \Delta(X\times \Delta(X))$ via,
$$F^n(E) = F(\xi_n^{-1}(E)).$$
That is, $F^n$ assigns the same probability to $(x,\mu^n)$ as $F$ assigned to $(x,\mu)$.
My question is: does $F^n$ converge to $F$. Intuitively, it seems as though it should, since $\mu^n$ is converging pointwise. However, I have gotten tripped up on these types of things before.
To make this work, you have to choose the $\mu_n$ in a "measurable way".
Let $X=\{0,1\}$, so that $\Delta(X)$ can be identified with $[0,1]$. Let $F$ be the product measure obtained from the uniform distributions on $X$ and $[0,1]$. In particular, the $\Delta(X)$-marginal of $F$ has full support $[0,1]$.
Let $N\subseteq [0,1]$ be non-measurable with $0\in N$. For $\mu\in N$, let $\mu_n=\mu$ for all $n$. For $\mu\in[0,1]\backslash N$, let $\mu_n=(n-1)/n \mu+1/n 0$. Now if $\xi_n$ were measurable for some $n$, so would be the function $\zeta_n:[0,1]\to[0,1]$ given by $\zeta_n(\mu)=\mu_n$ and also the function $\mu\mapsto \zeta_n(\mu)-\mu$. Then, the set of points mapped by the latter function to $0$ would be measurable. But that set is $N$. So $\xi_n$ is never measurable.
So in general, $F_n$ is not even defined.
Now assume that $\xi_n$ is always measurable and let $f:X\times\Delta(X)\to\mathbb{R}$ be bounded and continuous. Then $$\int f~dF_n=\int f~d F\circ\xi_n^{-1}=\int f\circ\xi_n~dF,$$ where the last equality follows from the change of variables formula for pushforward measures.
Since $f$ is bounded, the sequence of functions $\langle f\circ\xi_n\rangle$ is dominated by a constant function, which is of course integrable. Since $f$ is continuous, $\langle f\circ\xi_n\rangle$ converges pointwise to $f$. So by the dominated convergence theorem, $$\lim_{n\to\infty }\int f~dF_n=\int f~dF.$$ So $\langle F_n\rangle$ converges to $F$, if all $\xi_n$ are measurable.