Strong Markov Property: Having Trouble Understanding Proof Using Strong Markov

Question

I'm having trouble understanding this proof of Example 1.4.4 From Norris' Markov Chains: Let $X_n$ be a DTMC, with transition matrix P and state-space $I$. Let $Y_m=X_{T_m}$ for $m \in \mathbb{N}$. Show $Y_m$ is a DTMC.

Define $$T_0=\inf\{n\geq0:X_n\in J\subset I\}$$ and $$T_{m+1}=\inf\{n> T_{m}:X_n\in J\subset I\}$$

For $i_0,...,i_{m+1} \in J$ we have \begin{align*} &P(Y_{m+1}=i_{m+1}|Y_0=i_0, ..., Y_m=i_m)\\ &=P(X_{T_{m+1}}=i_{m+1}|X_{T_{0}}=i_0,...,X_{T_{m}} =i_m)\\ &=P(X_{T_{m+1}}=i_{m+1}|X_{T_m}=i_m)\\ &=P(X_{T_{1}}=i_{m+1}|X_{0}=i_m) \end{align*}

I'm confused by several lines of the proof. Does the second equality follow from the fact that $X_n$ is a DTMC and thus by Markov property? Does the third equality follow form the fact that $T_m$ is a stopping time so by Strong Markov Property Markov Chain regenerates at $T_m$ so its like starting from $0$ and hitting for first time. So shouldn't it be $X_{T_{0}}=i_{m+1}$ and not $X_{T_{1}}=i_{m+1}$? (Really confused about this).

Also I don't get how the proof shows that $Y_m$ is a Markov chain. We showed that $P(Y_{m+1}=i_{m+1}|Y_0=i_0, ..., Y_m=i_m)=P(X_{T_{1}}=i_{m+1}|X_{0}=i_m)$. How does this show that $Y_m$ is a Markov chain and has the Markov property?

Answer 1

• the first equality is the defimition of $Y_0, \ldots, Y_{m+1}$
• the second equality is the strong Markov property ($T_m$ is a stopping time)
• you could write $P(X_{T_1} = i_{m+1} \mid X_{T_0} = i_m)$ instead of $P(X_{T_1} = i_{m+1} \mid X_0 = i_m)$ if you wish; the probabilities are equal (both are the probability that the second visit to $J$ is $i_{m+1}$ given that the first visit to $J$ is at $i_0$).
• You've shown that the probability $P(Y_{m+1} = i_{m+1} \mid Y_0 = i_0, \ldots, Y_m = i_m)$ is a function of $i_{m+1}$ and $i_m$ (and in particular does not depend on $i_0, \ldots, i_{m-1}$), which proves the Markov property.