Consider the Markov chain on nonnegative integers that does the following: from any site $x \geq 0$ it jumps to $x + 1$ with probability $p$ and to 0 with probability $1 − p$. Is this chain transient? Null recurrent? Positive recurrent? You need to prove whatever you claim.
Let $S_0=x$ and $S_n=x+X_1+X_2+\dots +X_n$. $S_n$ is recurrent if $$P(\exists n\geq 1~|~ S_n=0)=1.$$ Furthermore, if the first return time is finite then we have positive recurrence and if it is infinite then we have null recurrence.
I believe that this depends on $p$. I am interested in $$P(\text{return to x} ~|~ \text{started at x})$$ I am having a hard time determining what I should do next and I was hoping for a bit of help.
I want to try and use a martingale. I am wondering if I need to know the mean and the variance of the Markov chain? I am not really sure how to find either of these.
Thank you for any help you can give to assist me.
I'm not sure how to use martingales in this instance but here is one way:
For any $i,j \in \mathbb{Z}_{\geq 0},$ we have $P^{(j+1)}_{i,j} \geq P_{i,0} \cdot P_{0,1} \cdot P_{1,2} \cdots P_{j-1,j} = (1-p)p^j > 0$. Similarly, $P^{(i+1)}_{j,i}>0$. Then all states $i,j$ communicate and so the MC is irreducible.
Next, note that $P^{(n)}_{0,0} = (1-p)\sum^{n-1}_{k=0} {n-1 \choose k}p^{n-1-k}(1-p)^k = (1-p)(1)^{n-1}=(1-p).$ Therefore,$\ \sum^{\infty}_{n=1}P^{(n)}_{0,0} = \sum^{\infty}_{n=1} (1-p) = \infty$ and so the MC is recurrent.
Finally, let $T_i$ be the waiting time for state $i$ and note that $P(T_0=k\ | \ X_0=0)=p^{k-1}(1-p).$ Then it follows that $E[T_0 \ | \ X=0] = \sum^{\infty}_{k=0}kp^{k-1}(1-p)=\frac{1-p}{p} \sum^{\infty}_{k=0}kp^k= \frac{1}{1-p}< \infty.$ So the MC is positive recurrent.