Can An irreducible Markov chain with infinitely many states be a positive recurrent chain or periodic chain?

Question

I understand that any irreducible finite Markov chain is necessarily positive recurrent, however can this be the case for a Markov chain with infinitely many states? Similarly for periodicity? There also seems to be a link between the 2, either both statements are true or false.

Answer 1

For infinite-state Markov chains, all $12$ combinations of (irreducible or not), (aperiodic or not), and (positive recurrent or null recurrent or transient) are possible.

There's also the further detail that if you're not irreducible, then different states of the Markov chain can vary in the other properties.

Irreducible, but periodic, and transient

Consider the Markov chain whose state space is the integers, where we go from $k$ to $k+1$ with probability $\frac23$ and from $k$ to $k-1$ with probability $\frac13$.

This is periodic: if you're in state $0$ at time $0$, then at time $t$ you can only be in a state $k$ if $k \equiv t \pmod 2$.

It is transient: the Markov chain has a drift rightward, and if you go from $k$ to $k+1$, there is a $\frac12$ chance you'll never return to $k$, so the expected number of visits to any state is finite.

Irreducible, but periodic, and null recurrent

Consider the Markov chain whose state space is the integers where we go from $k$ to $k+1$ with probability $\frac12$ and to $k-1$ otherwise (also with probability $\frac12$).

This Markov chain still has period 2, as above.

It is well-known to be null recurrent. You do actually visit every state infinitely often with probability $1$ - it's just that the expected number of steps between such visits is also infinite.

Irreducible, but periodic, and positive recurrent

Consider the Markov chain whose state space is the integers with the following transition probabilities:

$$ P_{k,k+1} = \begin{cases} \frac23 & k < 0 \\ \frac12 & k = 0 \\ \frac13 & k > 0 \end{cases} \qquad P_{k,k-1} = \begin{cases} \frac13 & k < 0 \\ \frac12 & k = 0 \\ \frac23 & k > 0 \end{cases} $$ This is still periodic for all the same reasons. It is positive recurrent because it actually has a stationary distribution: $\pi_k = \frac38 \cdot (\frac12)^{|k|}$ for $k \ne 0$, with $\pi_0 = \frac14$. Intuitively, there is a "drift towards the origin" that keeps the Markov chain from getting spread infinitely thin.

Versions of the above that are aperiodic

Take any of the $3$ previously constructed chains, and make a "lazy version": from each state, with probability $\frac12$ we stay in $k$, and with probability $\frac12$ we use the transition probabilities described above.

This is still transient/null recurrent/positive recurrent for the same reasons, but it is aperiodic, because the possibility of staying put for any number of steps breaks periodicity.

Versions of the above that are not irreducible

Take any of the $6$ previously constructed Markov chains, and then take two disjoint copies of the state space, with the same transition probabilities within each copy. This is not irreducible because it's impossible to get from one copy to the other.