On the definition of Markov process

Question

A stochastic process $x (t)$ is called Markov if for every $n$ and $t_{1}<t_{2} \ldots<t_{n},$ we have $$ \begin{array}{l} P\left(x\left(t_{n}\right) \leq x_{n} \mid x\left(t_{n-1}\right), \ldots, x\left(t_{1}\right)\right) \\ =P\left(x\left(t_{n}\right) \leq x_{n} \mid x\left(t_{n-1}\right)\right) \end{array} $$ This is equivalent to $$ \begin{array}{l} P\left(x\left(t_{n}\right) \leq x_{n} \mid x(t) \text { for all } t \leq t_{n-1}\right) \\ =P\left(x\left(t_{n}\right) \leq x_{n} \mid x\left(t_{n-1}\right)\right) \end{array} $$

How to show that equivalence ?

Answer 1

Suppose that $$\mathbb{P}(X(t_n) \leq x_n \mid X_t \, \, \text{for all $t \leq t_{n-1}$}) = \mathbb{P}(X(t_n) \leq x_n \mid X(t_{n-1})). \tag{1}$$

By the tower property of conditional expectation, we have

\begin{align*} &\mathbb{P}(X(t_n) \leq x_n \mid X(t_1),\ldots,X(t_{n-1})) \\&= \mathbb{E} \bigg[ \mathbb{E}\big(1_{X(t_n) \leq x_n} \mid X_t \, \, \text{for all $t \leq t_{n-1}$}\big) \mid X(t_1),\ldots,X(t_{n-1})) \bigg]. \end{align*}

Using $(1)$ for the inner conditional expectation, we see that

\begin{align*} \mathbb{P}(X(t_n) \leq x_n \mid X(t_1),\ldots,X(t_{n-1})) &= \mathbb{E} \bigg[ \mathbb{E}\big(1_{X(t_n) \leq x_n} \mid X(t_{n-1}) \big) \mid X(t_1),\ldots,X(t_{n-1}) \bigg] \\ &= \mathbb{E}(1_{X(t_n) \leq x_n} \mid X(t_{n-1})), \end{align*} i.e. $(X_t)_t$ is a Markov process (in the sense of your definition).

It remains to show that any Markov process satsfies $(1)$. To this end, recall the following characterization.

Lemma: Let $Y \in L^1$ and $\mathcal{A}$ a $\sigma$-algebra. Then $Z=\mathbb{E}(Y \mid \mathcal{A})$ if, and only if, $Z \in L^1$ is $\mathcal{A}$-measurable and $$\forall A \in \mathcal{A}\::\: \int_A Y \, d\mathbb{P} = \int_A Z \, d\mathbb{P}.$$

Consequently, by your definition of a Markov process, we have

$$\int_A 1_{\{X(t_n) \leq x_n\}} \, d\mathbb{P} = \int_A \mathbb{P}(X(t_n) \leq x_n \mid X(t_{n-1})) \, d\mathbb{P}$$

for any set $A$ of the form $A= \bigcap_{j=1}^{n-1} \{X(t_j) \in B_j\}$ with $B_j$ Borel. For fixed $t_{n-1} < t_n$ we define

\begin{align*} \mu(A) &:= \int_A 1_{\{X(t_n) \leq x_n\}} \, d\mathbb{P} \\ \nu(A) &:= \int_A \mathbb{P}(X(t_n) \leq x_n \mid X(t_{n-1})) \, d\mathbb{P} \end{align*}

for $A \in \mathcal{F}_{t_{n-1}} := \sigma(X_t; t \leq t_{n-1})$, then $\mu$ and $\nu$ are (finite) measures. Moreover, our earlier considerations shows that $\mu(A)=\nu(A)$ for any set $A$ of the form $A= \bigcap_{j=1}^{n-1} \{X(t_j) \in B_j\}$, where $0 \leq t_1 < \ldots < t_{n-2} < t_{n-1}$ are arbitrary and $B_j$ are any Borel sets. Since sets of this type are a generator of $\mathcal{F}_{t_{n-1}}$ which is stable under intersections, the uniqueness of measure theorem shows that $\mu(A)=\nu(A)$ for all $A \in \mathcal{F}_{t_{n-1}}$, i.e.

$$\int_A 1_{\{X(t_n) \leq x_n\}} \, d\mathbb{P} = \int_A \mathbb{P}(X(t_n) \leq x_n \mid X(t_{n-1})) \, d\mathbb{P}$$

for all $A \in \mathcal{F}_{t_{n-1}}$. Since we also know that $\mathbb{P}(X(t_n) \leq x_n \mid X(t_{n-1}))$ is $\mathcal{F}_{t_{n-1}}$-measurable, we conclude, by the above lemma, that

$$\mathbb{E}(1_{\{X(t_n) \leq x_n\}} \mid \mathcal{F}_{t_{n-1}}) = \mathbb{P}(X(t_n) \leq x_n \mid X(t_{n-1})).$$