markov chain on state {1, 2, 3, 4, 5, 6 , 7}

Question

markov chain on state {1, 2, 3, 4, 5, 6 , 7}

what are the probabilities of states 1 , 2 , and 4 in the stationary distribution of the Markov chain s shown in the image. The label to the left of an arrow gives the corresponding transition probability.

Answer 1

This is an discrete finite ergodic Markov Chain so there is a unique stationary distribution $\pi$ with $\sum \pi_i =1$. Here are three approaches:

1. Looking at the symmetries and shape of your diagram

   • $\pi_2=\pi_3$ and $\pi_4=\pi_5=\pi_6=\pi_7$.
   • To keep the distribution stationary, you need $0.4 \pi_1=0.2 \pi_2$ so $\pi_2=2\pi_1$;
   • you also have $0.4 \pi_2=0.2 \pi_4$ so $\pi_4 = 2\pi_2 =4\pi_1$.
   • Since $\sum \pi_i =1$, you get $\pi_1=\frac{1}{1+2\times 2+4\times4}=\frac{1}{21}$ and thus $\pi_2=\frac2{21}$ and $\pi_4=\frac4{21}$, which are about $0.047619$, $0.095238$ and $0.190476$

2. The transition matrix is $P=\left(\begin{matrix} 0.2 & 0.4 & 0.4 & 0.0 & 0.0 & 0.0 & 0.0 \\ 0.2 & 0.0 & 0.0 & 0.4 & 0.4 & 0.0 & 0.0 \\ 0.2 & 0.0 & 0.0 & 0.0 & 0.0 & 0.4 & 0.4 \\ 0.0 & 0.2 & 0.0 & 0.8 & 0.0 & 0.0 & 0.0 \\ 0.0 & 0.2 & 0.0 & 0.0 & 0.8 & 0.0 & 0.0 \\ 0.0 & 0.0 & 0.2 & 0.0 & 0.0 & 0.8 & 0.0 \\ 0.0 & 0.0 & 0.2 & 0.0 & 0.0 & 0.0 & 0.8 \end{matrix}\right)$

   • $P^n$ converges as $n$ increases, and the rows of the limit give you the staionary distribution.
   • For example, the top row of $P^{100}$ is 0.04761905 0.09523810 0.09523810 0.1904762 0.1904762 0.1904762 0.1904762 close to the exact result while the second row is 0.04761905 0.09746764 0.09300855 0.1958588 0.1958588 0.1850936 0.1850936 is a little further away and might need a higher value of $n$

3. Consider the left eigenvalues and left eigenvectors of $P$ (your computer system might prefer the eigenvalues and eigenvectors of $P^T$)

   • The left eigenvalues are $1, \frac25(1+\sqrt2), \frac45,\frac45,-\frac25, \frac25, \frac25(1-\sqrt2)$.
   • My computer gives the left eigenvector associated with $1$ as $\left(\frac{1}{\sqrt{73}}, \frac{2}{\sqrt{73}}, \frac{2}{\sqrt{73}}, \frac{4}{\sqrt{73}}, \frac{4}{\sqrt{73}}, \frac{4}{\sqrt{73}}, \frac{4}{\sqrt{73}}\right)$ where the sum of squares is $1$
   • The values of that eigenvector do not sum to $1$ without squaring, but rescaling gives $\left(\frac{1}{21}, \frac{2}{21}, \frac{2}{21}, \frac{4}{21}, \frac{4}{21}, \frac{4}{21}, \frac{4}{21}\right)$ which does sum to $1$ and so is the stationary distribution