recurrence criterion for random-walk like Markov chain

698 Views Asked by At

Suppose we have a random-walk like Markov chain, i.e. state space is the set of all integers from $-\infty$ to $\infty$, the transition probability $P_{ij}$ is nonzero only when $j=i+1$ or $j=i-1$.

The difference from the random walk is that the transition probability is periodic in state space i.e. $P_{i,i+1}=p_1,P_{i+1,i+2}=p_2,\dots,P_{i+L-1,i+L}=p_L,P_{i+L,i+L+1}=p_1$, and none of $p_i$ is 0 or 1.

My question is: under what conditions (in terms of $\{p_i\}$) is this Markov chain recurrent or transient?

1

There are 1 best solutions below

3
On BEST ANSWER

For every $x$, let $T_x$ denote the first hitting time of $x$. Looking at the chain at its successive visits of multiples of $L$, one sees that there is recurrence if and only if $h_L=\frac12$, where, for $0\leqslant x\leqslant 2L$, $$ h_x=P_x(T_{2L}\lt T_0). $$ The computation of $h_L$ involves the finite Markov chain on $\{0,1,\ldots,2L\}$ hence the standard approach applies, which is as follows.

Let $r_x=r_{x+L}=p_{x+1}$ for $0\leqslant x\leqslant L-1$. By the Markov property, the vector $(h_x)_{0\leqslant x\leqslant 2L}$ solves the linear system $$ h_{2L}=1,\quad h_0=0,\quad h_x=r_xh_{x+1}+(1-r_x)h_{x-1}\quad (1\leqslant x\leqslant 2L-1). $$ The change of variable $h_x=\sum\limits_{y=1}^xk_y$ yields $r_xk_{x+1}=(1-r_x)k_x$ for every $1\leqslant x\leqslant 2L-1$, which is easily solved. The normalization $h_{2L}=1$ yields finally $$ h_x=\frac{K_x}{K_{2L}},\qquad K_x=\sum\limits_{y=1}^xR_{y-1},\qquad R_{y}=\prod_{z=1}^{y}\frac{1-r_z}{r_z}. $$ Up to this point, this is entirely general and applies to every random walk on the integers with transitions $(r_x)$. We now turn to the specifics of the case at hand.

Due to the periodicity of $(r_x)$, for every $1\leqslant y\leqslant L$, $R_{y+L}=R_LR_y$ hence $$ K_{2L}-K_L=\sum\limits_{y=L+1}^{2L}R_{y-1}=\sum\limits_{y=1}^{L}R_{y+L}=R_L\sum\limits_{y=1}^{L}R_{y}=R_LK_L, $$ which yields the simple identity $$ h_L=\frac1{1+R_L},\qquad R_L=\prod_{x=1}^L\frac{1-p_x}{p_x}. $$ This shows that:

  • if $R_L=1$, then the chain is null recurrent,
  • if $R_L\lt1$, then $h_L\gt\frac12$ hence the chain is transient (and goes to $+\infty$ at linear speed),
  • if $R_L\gt1$, then $h_L\lt\frac12$ hence the chain is transient (and goes to $-\infty$ at linear speed).