Exercise from Norris' book on Markov chains

Question

Let $(X_n)$ be a Markov chain on $\mathbb{N}$ with transition probabilities satisfying:

$$p_{0,1}=1,\quad p_{i,i-1}+p_{i,i+1}=1,\quad p_{i,i+1}=\left(\frac{i+1}{i}\right)^{\alpha}p_{i,i-1}$$

The exercise asks to find $\mathbb{P}(X_n\to \infty\;\text{as}\;n\to\infty)$

The problem is that I am having real trouble getting started off, because I cannot express the event $X_n\to\infty$ in a more suitable form. Any help on this would be greatly appreciated. My first thought was:

$$\mathbb{P}(X_n\to\infty)=\prod_k\mathbb{P}(X_n\geq k\;\text{eventually})=\prod_k \mathbb{P}(X_n \neq 0,\cdots X_n\neq k-1\;\text{eventually})$$

But I am having trouble with translating "eventually".

Answer 1

As often, it is difficult to answer this question because the OP says nothing about their background. Anyway, a standard approach to determine the recurrence/transience of a Markov chain (its type) is to compute $P_1(T_0\,\text{infinite})$ as the limit of $P_1(T_n\lt T_0)$ when $n\to\infty$, since each $P_1(T_n\lt T_0)$ involves only a finite Markov chain.

In the case at hand, for each positive $n$, the Markov property after one step shows that the quantities $u_i=P_i(T_n\lt T_0)$ solve the system $u_0=0$, $u_n=1$, and, for every $1\leqslant i\leqslant n-1$, $$u_i=p_{i,i+1}u_{i+1}+p_{i,i-1}u_{i-1},$$ Assume furthermore that, as in the case at hand, there exists some positive sequence $(w_i)_{i\geqslant1}$ such that, for every $i\geqslant1$, $$w_{i+1}p_{i,i+1}=w_ip_{i,i-1},$$ then $$\frac{u_{i+1}-u_i}{w_{i+1}}=\frac{u_i-u_{i-1}}{w_i},$$ that is, $$u_{i+1}=u_i+\frac{w_{i+1}}{w_1}u_1,$$ hence, $$u_i=\frac{u_1}{w_1}\sum_{k=1}^iw_k.$$ The limit condition $u_n=1$ shows finally that $$u_1=\frac{w_1}{\sum\limits_{k=1}^nw_k}.$$ Thus, when $n\to\infty$, $$P_1(T_n\lt T_0)=\frac{w_1}{\sum\limits_{k=1}^nw_k}\to\frac{w_1}{\sum\limits_{k=1}^\infty w_k}=P_1(T_0\,\text{infinite}).$$ In the exercise, $w_i=i^{-a}$ hence all this yields:

• If the series $\sum\limits_kw_k$ diverges, then $P_1(T_0\,\text{infinite})=0$, that is, the Markov chain is recurrent. When $w_i=i^{-a}$ for every $i$, this happens if and only if $a\leqslant1$.

• If the series $\sum\limits_kw_k$ converges, then $P_1(T_0\,\text{infinite})\ne0$, that is, the Markov chain is transient. When $w_i=i^{-a}$ for every $i$, this happens if and only if $a\gt1$.

Nota: The probabilistic structure hidden behind these furious computations (particularly simple in the present case) is called an electric network, see the small book Random walks and electric networks by Doyle and Snell for a masterful exposition of the subject.