Markov chain question

140 Views Asked by At

Consider an irreducible, recurrent Markov Chain ($X_n$) on a countable state space $S$ with transition probability $p(x,y).$ Pick a sigma-algebra $A \subset S$ and let $T_k=\inf\{n>T_{k-1}:X_n \in A\}$, with $T_0=0$, be the time of the $k$-th return time to $A$. (All $T_k<\infty$ by recurrence.) Define $Z_n=X_{T_n}$ for $n \geq 0$. Prove that, for $X$ started from any $x \in A$, $Z_n$ is a Markov chain on $A$ with a transition probability $Q$. Determine $Q$ and prove that $Z_n$ is irreducible and recurrent. Any help would be welcomed.

So basically we want to prove that $P^{\mu}(X_{T_{n+1}}=x|X_{T_{n}},X_{T_{n-1}},\ldots)=P^{\mu}(X_{T_{n+1}}=x|X_{T_{n}})$. I tried to use the Strong Markov Property, namely $P^{\mu}(1_B \circ\theta^{T_n}|F_{T_n})=P^{X_{T_n}}(B)$, I think this is the key, but I do not know how. I do not have any ideas on how to relate $P$ to $Q$. Any hints would be sufficient. Thank you.