Example of a markov chain that has a distribution that converges to some limit.

Question

Can someone give me an example of a Markov chain that has a distribution that converges to some limit which depends on the initial distribution?

Answer 1

The "main theorem" of this subject says that an aperiodic irreducible Markov chain on a finite state space converges to a unique stationary distribution regardless of the initial distribution. (To my knowledge this theorem has no name, but it is closely related to the Perron-Frobenius theorem from linear algebra.)

If you drop the aperiodic assumption, then you still have a unique stationary distribution but you might not observe convergence to it. If you drop the irreducible assumption, then you can have non-unique stationary distributions, but the actual way that this happens is rather simple. Basically, a reducible Markov chain is some number of separated irreducible Markov chains on disjoint subsets of the state space, called communicating classes. The different stationary distributions are entirely parametrized by the total probability assigned by the initial distribution to each communicating class.

So for example, you could have a chain with transition matrix

$$\begin{bmatrix} 1/4 & 3/4 & 0 & 0 \\ 1/2 & 1/2 & 0 & 0 \\ 0 & 0 & 1/3 & 2/3 \\ 0 & 0 & 4/5 & 1/5 \end{bmatrix}.$$

There is a stationary distribution on this chain which is concentrated on $\{ 1,2 \}$, namely $\pi_1=\begin{bmatrix} 2/5 & 3/5 & 0 & 0 \end{bmatrix}$. There is another which is concentrated on $\{ 3,4 \}$ namely $\pi_2=\begin{bmatrix} 0 & 0 & 6/11 & 5/11 \end{bmatrix}$. Any convex combination of these two is a stationary distribution. The limiting distribution will be $(p_0(1)+p_0(2))\pi_1+(p_0(3)+p_0(4))\pi_2$.

Answer 2

Just take any non-irreducible Markov chain with two communicating classes, both of which satisfy the hypotheses of the ergodic theorem. Then the limit distribution will depend on which communicating class one starts in (even though any convex combination of those two limiting distributions satisfies the requirement of stationarity).

EDIT: basically look at Ian's answer above. Each block in the block diagonal matrix corresponds to a different communicating class; each one has a separate corresponding limiting distribution, which one occurs depends on where in the Markov chain one starts.