Bayesian statistics and convergence to "the true distribution"

103 Views Asked by At

Suppose we have a random variable $\mathbb X : \Omega \to E$ and let's define random variables $\mathbb Y_1, \mathbb Y_2, \mathbb Y_3, \ldots, $ all i.i.d to $\mathbb X$. Now we choose an arbitrary distribution $\mathbb Z_0$ with the same range as $\mathbb X$, called our prior.

Now we define our posterior random variable $\mathbb Z_1$ via Bayesian adjustment according to Bayes' law observing $\mathbb Y_1$. Consider $\mathbb Z_1$ to be our prior. Repeat this and define a sequence $\left\{\mathbb Z_n \right\}_{n=1}^\infty$.

Does it make sense to talk about a limit whereas $\mathbb Z_n \to \mathbb X$ in probability, meaning that for all $\varepsilon >0$ we have $\displaystyle\lim_{n\to\infty} \mathbb P(\mathbb| Z_n - \mathbb X| < \varepsilon) = 1$? Does a stochastic random variable $\mathbb Z_0$ always exist?