Most Markov chain definitions are false

674 Views Asked by At

When introducing discrete, time-homogenouos Markov chains $(X_n)_{n\geq 0}$, a lot of introductory lecture notes simply seem to assume the existence of a probability measure $\mathbb{P}$ on the common domain $\Omega$ of the $X_n$, given the distribution of $X_0$ and the transition matrix $T$.

(We need $\mathbb{P}$ to talk about things like $$\mathbb{P}(X_7=u \land X_{23}=v)$$ resp. $$\mathbb{P}((X_n)_{n\geq 0}=(s_n)_{n\geq 0}),$$for some $u,v$ from the state space $S$ resp. a sequence of values $(s_n)_{n\geq 0}$ ins $S$, a measure $\mathbb{P}$ is required.)

Only one set of lecture notes , out of many that I've consulted, have in passing mentioned that there is a thing such as the Ionescu-Tulcea theorem that shows that such probability measure $\mathbb{P}$ indeed exist (hence the title). This theorem is interestingly is not yet on Wikipedia - only on the german Wikipedia.The theorem is way above my head to understand it, as it is formulated.

My questions are:

  • Do we really need this theorem? If most lectures notes gloss over it, perhaps it is trivial that $\mathbb{P}$ exists?

  • Since the Ionescu-Tulcea theorem seems to apply for general Markov chains, does its statement (which I don't fully understand currently) and proof perhaps simplify significantly for discrete, time-homogenuous Markov chains (perhaps if we additionally assume a finite state space)? I'd be very happy, if I could understand it's proof.

1

There are 1 best solutions below

3
On BEST ANSWER

Suppose your goal is to construct a discrete-time stochastic process $(X_n)_{n\ge 0}$, each random variable taking values in a measurable space $(E,\mathcal E)$. The canonical approach is to construct a probability measure $\Bbb P$ on the sequence space $E^{\{0,1,2,\ldots\}}$ endowed with the product $\sigma$-fields, using the coordinate maps $X_n(\omega)$ to realize the process.

The Kolmogorov Extension Theorem asserts the existence (and uniqueness) of such a $\Bbb P$ with finite-dimensional distribution given by a consistent family of probability measures provided those probability measures are "inner regular" (which means in particular that $E$ is a topological space with associated Borel $\sigma$-field $\mathcal E$...) One simple sufficient condition guaranteeing this is that every probability measure on $(E,\mathcal E)$ is inner regular; for example $(E,\mathcal E)$ could be a standard Borel space. This regularity condition on $(E,\mathcal E)$ is met if $E$ is countable and $\mathcal E$ is the power set of $E$, as it will be for a discrete-space Markov chain.

The Ionescu-Tulcea theorem imposes no regularity condition on $(E,\mathcal E)$. The trade-off is that in the I-T theorem the finite-dimensional distributions need to be specified in a certain way by a family of conditional distributions. Not every consistent family of finite-dimensional distributions can be specified in this way (unless the state space is sufficiently regular—circle back to the Kolmogorov theorem). But in the case of a Markov chain (on a general state space) the conditional distributions required by I-T can be defined once the initial distribution and the one-step transition probability kernel is given.

In short, Ionescu-Tulcea applies to give a construction of any discrete-time Markov chain. Kolmogorov applies when the state space of the Markov-chain-to-be satisfies a mild regularity condition. Both apply when the state space is countable.

To answer your final question, I don't think that the proof of I-T is simpler when the state space is discrete. As there is no topological aspect to the proof, it is somewhat simpler than the proof of the Kolomogorov theorem.