Markov chain with transitions from state $i$ to $i+1$ with prob $1-\frac{1}{2i^\alpha}$

Question

A Markov chain $\{X_n,n\geq0\}$ has state space $\{0,1,2,...\}$ and transitions from each $i$ to $i+1$ with probability $1-\frac{1}{2i^\alpha}$ and to $0$ with probability $\frac{1}{2i^\alpha}$. It transitions from $0$ to $1$ with probability $1$. Show that the chain is irreducible. Determine if $0$ is transient or recurrent depending on $\alpha$.

Given two states $i$ and $j$, we can find a path that goes from $i$ and $j$ and one that goes from $j$ to $i$, thus we only have one class and the chain is irreducible.

Now, let $T_0$ be the return time of $0$ (i.e. first time we hit $i$ since we leave it). Then the state $i$ is transient if $Pr(T_i<\infty)=1$ and recurrent if $Pr(T_i<\infty)<1$.

If $f_{00}(n)=Pr(T_0=n)$, we have that $f_{00}(1)=0,\ f_{00}(2)=\frac{1}{2}$, and for $n>2$, $\ f_{00}(n)=\frac{1}{2(n-1)^\alpha}\prod_{i=1}^{n-2}(1-\frac{1}{2i^\alpha})$.

$Pr(T_i<\infty)=\sum_{n\geq1}f_{00}(n)=\frac{1}{2}+\sum_{n\geq3}\frac{1}{2(n-1)^\alpha}\prod_{i=1}^{n-2}(1-\frac{1}{2i^\alpha})$

But I don't know how to compute this sum. Could you help me? Is there an easier way to solve the problem? Thanks in advance!

Answer 1

Equivalently, you can compute $\Pr(T_0=\infty)$, and then the state $0$ will be recurrent if and only if this probability is $0$. Note that by continuity of measure, \begin{equation} \Pr(T_0=\infty)=\lim_{n\to \infty} \Pr(T_0\geq n+1)=\lim_{n\to \infty} \prod_{i=1}^n \left( 1-\frac{1}{2i^{\alpha}}\right). \end{equation} It is a standard result that this sequence converges to a nonzero number if and only if $\sum_{i=1}^{\infty} \frac{1}{2i^{\alpha}}<\infty$. Of course, this occurs if and only if $\alpha>1$ by the usual divergence of the harmonic series. As a sanity check, larger $\alpha$ makes it less likely we return to $0$. Therefore, this random walk will be recurrent if and only if $\alpha\leq 1$.

Answer 2

This is a birth-death process, so we can use the detailed balance equations to determine the stationary distribution (assuming the process is positive recurrent). From $$ \pi_0 = \sum_{n=1}^\infty \frac1{2n^\alpha} $$ we have that $\pi_0<1$ if and only if $\sum_{k=1}^\infty k^{-\alpha}<2$. So my intuition leads me to believe that this condition implies the chain is positive recurrent, strict equality implies that it is null recurrent, and the reverse inequality implies that the chain is transient. This is not a formal proof, but hopefully lends some insight (and is too long for a comment).