How to apply the strong Markov property in this case?

182 Views Asked by At

I'm trying to understand the following proof:

Theorem: Let $(X_n)$ be an irreducible $(\alpha, \mathbf p)$-Markov chain with a finite state space $S$. Then $(X_n)$ is positive recurrent.

Proof: Let $i\in S$. With $T_i:=\inf\{n\geq 1: X_n=i\}$ and since $(X_n)$ is irreducible: $$\forall j\in S \ \ \exists k_j\in \mathbb N : \mathbb P_j(T_i \leq k_j)>0.$$ Therefore it follows with $K:=\max\{k_j:j\in S\}$ that $$\exists\varepsilon>0 \ \ \forall j \in S: \mathbb P_j(T_i \leq K)\geq \varepsilon.$$ From the strong Markov property $\color{red}{\text{follows}}$ $$\forall j \in S \ \ \forall n\in \mathbb N : \mathbb P_j(T_i > nK)\leq(1-\varepsilon)^n.$$ Therefore: $$\mathbb E_i(T_i)=\sum_{n\geq 0} \mathbb P(T_i>n)\color{red}{\leq} K \sum_{n\geq 0} \mathbb P(T_i>nK) < \infty.$$

The parts I don't understand are colored in red. I tried to find this particular proof online hoping it would contain the omitted steps but couldn't, it doesn't seem to very standard.

The question is mainly about the first issue with applying the strong Markov property yet I'd very much appreciate if the second will be clarified as well (which I guess is a one-liner).

1

There are 1 best solutions below

2
On BEST ANSWER

First part:

For every $n$, the simple Markov property at time $nK$ yields $$P_j(T_i\gt(n+1)K\mid T_i\gt nK,X_{nK}=k)=u(k),\qquad u(k)=P_k(T_i\gt K),$$ and $u(\ )\leqslant1-\varepsilon$ uniformly by hypothesis hence $$P_j(T_i\gt(n+1)K,X_{nK}=k)\leqslant(1-\varepsilon)P_j(T_i\gt nK,X_{nK}=k).$$ Summing these over $k$ yields $$P_j(T_i\gt(n+1)K)\leqslant(1-\varepsilon)P_j(T_i\gt nK),$$ which implies the desired upper bound.

Second part:

For every nonnegative integer valued random variable $S$ and every $mK\leqslant n\lt(m+1)K$, $$P(S\gt n)\leqslant P(S\gt mK).$$ Decomposing the first sum below into subsums of $K$ terms, one gets $$E(S)=\sum_{n\geqslant0}P(S\gt n)=\sum_{m=0}^\infty\sum_{n=mK}^{(m+1)K-1}P(S\gt n)\leqslant\sum_{m=0}^\infty\sum_{n=mK}^{(m+1)K-1}P(S\gt mK)=K\sum_{m=0}^\infty P(S\gt mK).$$