How to prove the existence of the limit of Markov transition matrix?

Question

Does the limit of a Markov transition matrix $M$: $$\lim_{n\to\infty}M^n$$ always exist? And if yes, how to prove it?

Answer 1

The limit does not always exist. The simplest counterexample is $M=\begin{pmatrix}0&1\\1&0\end{pmatrix}$, where $M^{2n}=I$ and $M^{2n+1}=M$ for every natural number $n$. However, the limit exists if all entries of $M$ are positive. This is a consequence of the Perron-Frobenius Theorem, whose proof is no trivial matter.

Answer 2

The limit does not always exist. You can see it just by consider an example, say take $M$ to be the $2\times 2$ matrix with $0$ on the main diagonal and $1$ off the diagonal. In the context of Markov chains it is instructive to understand why you can't expect convergence always. When convergence occurs, it means there is a stationary distribution, and thus that the stochastic behaviour in the long run becomes stable. So now, imagine there are two states and the probability of passing from one state to the other is $1$. Clearly, no stability can ever be achieved. Intuitively, this is because the system is not connected enough or does not mix states well enough, in some sense. This can be made precise, as if all entries of the Markov transition matrix are positive, then it can be shown that the limit does indeed exist.