test a Markov Matrix for a stationary distribution.

Question

Consider a Markov chain with transition matrix

$ P = \begin{bmatrix} 1/2&1/2&0\\ 1/5&4/5&0\\ 0&0&1 \end{bmatrix} $

How many stationary distributions does this chain have?

Is it possible to answer this question without solving $\pi P = \pi$?

Answer 1

No need to compute powers of $P%$. Stationary distribution $(a,b,c)$ is obtained by solving $(a,b,c)P=(a,b,c)$. In this case you get $a=b$. $(\frac 2 7, \frac 5 7,0)$ and $(0,0,1)$ are stationary distributions and the most general stationary distribution is of the type $t((\frac 2 7, \frac 5 7,0)+(1-t)(0,0,1)$ where $t\in [0,1]$.

Answer 2

The matrix is block diagonal, so we can examine the blocks separately. $(0,0,1)$ is obviously a left eigenvector with eigenvalue $1$, so there’s one stationary distribution.

The upper-left block is itself row-stochastic, so we know that it, too, has $1$ as an eigenvalue. Its other eigenvalue is equal to its determinant, $\frac3{10}$, so there’s at most one other stationary distribution. The Perron-Frobenius theorem guarantees that we can find an eigenvector of $1$ with strictly positive entries, so this submatrix has exactly one stationary distribution.

It’s not very hard to compute this second stationary distribution directly if necessary, but we can also try out some simple guesses to come at it indirectly: $\frac45-\frac12 = \frac3{10}$, so based on the second row we might guess that $\left(-\frac52,1\right)^T$ is a right eigenvector of the submatrix. Sure enough, $-\frac52\cdot\frac12 + \frac12 = -\frac34 = \frac3{10}\cdot\left(-\frac52\right)$, so this checks out. Now, right eigenvectors are orthogonal to left eigenvectors of other eigenvalues, which means that $\left(1,\frac52\right)$ is a left eigenvalue of $1$, which we normalize to the stationary distribution $\left(\frac27,\frac57\right)$ for the upper-left submatrix.