Can we always find out the equilibrium distribution of a Markov chain

1k Views Asked by At

I am confused about whether the equilibrium distribution can always be calculated of a Markov Chain, if it is not, then in what situation can the equilibrium distribution be found out?

Also, for example, if an equilibrium distribution is $\pi = (\pi_0,\pi_1,\pi_2)$ can we say that the limit of the transition matrix is $\begin{pmatrix} \pi_0&\pi_1&\pi_2\\ \pi_0&\pi_1&\pi_2\\ \pi_0&\pi_1&\pi_2\\ \end{pmatrix}$?

1

There are 1 best solutions below

2
On BEST ANSWER

Let the transition matrix of the Markov chain be denoted by $P$. There is always an equilibrium distribution. One way to see this is to fix a probability vector $\mu$ and consider the sequence $$\mu_{N} = N^{-1} \sum_{j = 1}^{N} P^{j} \mu.$$ This is a bounded sequence of vectors so it has a convergence subsequence. I leave it to you to check $\lim_{N \to \infty} P\mu_{N} = \lim_{N \to \infty} \mu_{N}$ so that if $\nu = \lim_{N \to \infty} \mu_{N}$, then $P \nu = \nu$.

If $P$ is the transition matrix, the sequence $(P^{n})_{n \in \mathbb{N}}$ need not converge in general. For example, if $P = \left(\begin{array}{c c} 0 & 1 \\ 1 & 0 \end{array}\right)$, then $P^{2n} = \text{Id}$ and $P^{2n + 1} = P$. This is in spite of the fact that there is an equilibrium distribution, namely $\pi = (\frac{1}{2},\frac{1}{2})$. For irreducible finite state Markov chains, the sequence $(P^{n})_{n \in \mathbb{N}}$ converges to the matrix whose columns equal an equilibrium distribution if and only if the chain is aperiodic.

All of this is covered in the first chapter of Levin, Peres, and Wilmer's book Markov Chains and Mixing Times, available at Yuval Peres's website.