Periodic Markov Chains

Question

What's the reason behind why sometimes an irreducible, periodic markov chain still converges, with some good choice of an initial distribution? Conceptually, this doesn't make sense to me, because it should always oscillate, right? Is there a good way to see why (and when) this could be true? What does the initial distribution have to satisfy in order for the periodic chain to converge?

Answer 1

Simple example: $P=\begin{bmatrix} 0 & 1 \\ 1 & 0 \end{bmatrix}$. The stationary distribution is uniform, but it's not ergodic because once you know the initial state you know where the process will be at any future time (depending on whether the time is even or odd). On the other hand, if you start at exactly the stationary distribution, then of course you have convergence, because the sequence of distributions is just constant.

In higher dimensions, the situation can be a bit less trivial. In general a periodic Markov chain's transition matrix has one or more eigenvalues which are not $1$ yet have magnitude $1$. It could happen that your initial distribution happens to have no component in the direction of those eigenvectors, in which case there is no component of the distribution available to oscillate forever. For example, with $P=\begin{bmatrix} 0 & 1 & 0 \\ 1 & 0 & 0 \\ 1/3 & 1/3 & 1/3 \end{bmatrix}$, if your initial distribution assigns the same probability to state $1$ as to state $2$, then the distributions will converge, otherwise they won't. Note that for instance $\begin{bmatrix} 1/3 & 1/3 & 1/3 \end{bmatrix}$ is not actually stationary but if that is your initial distribution then you will see convergence, because the problematic right eigenvector is in the direction $\begin{bmatrix} 1 \\ -1 \\ 0 \end{bmatrix}$.

This last example was reducible but that is not essential, for example $P=\begin{bmatrix} 0 & 0 & 1 \\ 0 & 0 & 1 \\ 1/2 & 1/2 & 0 \end{bmatrix}$ is also periodic (for example, a path from 1 to 1 is always an even length).