Recurrent Markov chain on the integers with steps $+2$, $0$ and $-1$

192 Views Asked by At

I'd like to show you the (incomplete) solution I found to the following problem.

Let $X_n$ be a Markov chain with $\mathbb{Z}$ as state space and transition matrix given by $p_{i,i+2}=p$, $p_{i,i}=r$, $p_{i,i-1}=1-p-r$.

Define the subchain $Y_n$ that observes the preceding chain only when it moves, and determine its transition matrix;

Classify the states and determine the values of $p$ and $r$ for which the two chains are recurrent.

Solution

The first step is quite simple, and as said in Norris it amounts to calculate

$\tilde{p}_{i,j}=p_{i,j}/ \sum_{k\neq i}p_{i,k}$,

which yields

$\tilde{p}_{i,i+2}=\frac{p}{1-r}\quad\tilde{p}_{i,i-1}=\frac{1-p-r}{1-r}$.

For the second step I tried to follow the path traced by Norris in order to prove for which values of $p$ and $q$ the simple random walk on $\mathbb{Z}$ ($p_{i,i-1}=q, p_{i,i+1}=p$) is recurrent. Note that the chain just defined is irreducible, so all its states are either recurrent or transient. A general state $i$ is recurrent iff $\sum_{n=0}^\infty p_{i,i}^{(n)}=\infty$ (Norris 1.5.3). We have

$\sum_{n=0}^\infty\tilde{p}_{i,i}^{(n)}=\sum_{n=0}^\infty\tilde{p}_{i,i}^{(3n)}=\sum_{n=0}^\infty\binom{3n}{n}\big(\frac{p}{1-r}\big)^n\big(\frac{1-p-r}{1-r}\big)^{2n}$

and the general term of the last series is asintotic, using Stirling's formula, to

$\frac{1}{A\sqrt{\frac{2n}{3}}}\big[\frac{27}{4}\frac{p}{1-r}\big(\frac{1-p-r}{1-r}\big)^2\big]^n$

where $A=\sqrt{2\pi}$. So the series diverges iff $\frac{27}{4}\frac{p}{1-r}\big(\frac{1-p-r}{1-r}\big)^2\geq1$.

But I'm not o sure about it, it seems that there are not such $p$ and $r$ that the previous disequality is satisfied (the inequality $p+r<1$ has to be satisfied too...). Besides, I'm wondering how to proceed with the original chain $X_n$...