Markov chains exist

22 Views Asked by At

I'm reading through "Introduction to Stochastic processes". In page 6, the authors state

It can be shown that given any transition function $P$ and any initial distribution $\pi_0$, there is a probability space and random variables $X_n$, $n \ge 0$, defined on that space [such that they] form a Markov chain having transition function $P$ and initial distribution $\pi_0$.

I'm trying to figure out how to prove this formally. Let $S$ be the countable state space, with the powerset $\sigma$-algebra. A somewhat obvious candidate for the event space $\Omega$ is the set of functions $\mathbb N \to S$, representing the "paths" through the event space. The random variables $X_n$ could be taken as the projection functions $\Omega \to S$ with $$X_n(f) = f(n).$$ For these to be measurable functions, the $\sigma$-algebra for $\Omega$ would have to include all cylinder sets $X_n^{-1}\{s\}$. So we might take precisely the $\sigma$-algebra generated by these.

At this point, I want to apply Carathéodory's extension theorem to define the probability measure on $\Omega$. If $S$ is finite, then the set of intersections of finitely many cylinder sets is a semi-algebra. So we could define the probability of each of these sets as "whatever Markov chains tell us it needs to be". But I don't know how to prove $\sigma$-additivity for this semi-measure. If $S$ is countable then the set of cylinders isn't even a semi-algebra, so I'm lost on what to do.