Countable State Markov Chain reduciblity

Question

A countable state Markov chain has transition probabilities $$P(0, j) = p(j)$$ $$P(i, j) = p(j - i + 1) \quad i \geq 1 \quad j \geq i - 1$$

Show that if $p(0) = 0$ or $p(0) + p(1) = 1$ the chain is not irreducible and that if $p(0) > 0$ and $p(0) + p(1) \ne 1$ the chain in irreducible.

I think I understand intuitively why this is, but not sure what a proof should look like.

Answer 1

Note that $\ P(0,j) = p(j)\ $ implies that $\ \sum_\limits{j=0}^\infty p(j)=1\ $, and then $\ P(i,j)=p(j-i+1)\ $ for $\ j\ge i-1\ge0\ $ implies that $\ P(i,j)=0\ $ for $\ 0\le j\le i+2\ $.

• If $\ p(0)=0\ $, then $\ P(i, 0)=0\ $ for all $\ i\ $, so state $\ 0\ $ is inaccessible from any state. Consequently the chain is reducible.
• If $\ p(0)+p(1)=1\ $, then $\ p(j)=0\ $ for $\ j\ge2\ $, and so $\ P(1,i)=P(0,i)=0\ $ for all $\ i\ge2\ $. Therefore, no state $\ j\ $ is accessible from states $\ 0\ $ or $\ 1\ $, and the chain is again reducible.

• If $\ p(0)>0\ $, then $\ P(i,i-1)=p(0)>0\ $ for all $\ i\ge1\ $, so state $\ i-1\ $ is accessible from state $\ i\ $, and since accessibility is transitive, it follows that state $\ j\ $ is accessible from state $\ i\ $ for all $ j\in\{0,1,\dots,i-1\}\ $.

  If $\ p(0)+p(1)<1\ $ also, then there must be some integer $\ s\ge2\ $ such that $\ p(s)>0\ $. Since $\ P(i,i+s-1)=p(s)>0 $, state $\ i+s-1\ $, and, by induction, any state $\ i+n(s-1)\ $ for $\ n\ge1\ $, is accessible from state $\ i\ $. Now if $\ j\ge i\ $, then $\ i+n(s-1)>j\ $ for some $\ n\ge1\ $. Then state $\ i+n(s-1)\ $ is accessible from state $\ i\ $, and state $\ j\ $ is accessible from state $\ i+n(s-1)\ $ by the argument above, so state $\ j\ $ is accessible from state $\ i\ $.

  Thus if both $\ p(0)\ne0\ $ and $\ p(0)+p(1)\ne1\ $, then every state is accessible from every other and the chain is irreducible.