How to parameterize a set of stationary distributions for a Markov Chain?

77 Views Asked by At

Find the set of stationary distributions for: $$ P=\begin{pmatrix} 0&1/4&0&0&3/4\\ 3/4&0&0&0&1/4\\ 0&0&1&0&0\\ 0&0&0&1&0\\ 1/4&3/4&0&0&0\\ \end{pmatrix} $$

The answer was vector $\pi =(a, a, b, c, a)$ for when $3a+b+c=1$ and $a,b,c \geq 0$. I've only learned how to solve these by $\pi P=\pi$ in class, for which we would get a unique solution, then normalize $\pi$ by the sum of all the elements in $\pi$, that is $$ \pi_{ij}=\pi^*_{ij}\frac{1}{\sum_{i=1}^{5}\pi_i} $$ But I'm not sure how to approach the general/multi-solution case because while I've solved linear equations with multiple solutions before (set $\pi_3, \pi_4$ to a paramter; write solution as a linear combination) Idk how to address the fact we'd have a normalizing step at the end. Also, in this case I noticed rows 1 and 5 suggest different equations for $\pi_2$. Essentially, my linear algebra skills are not working in this application to Markov chains.

1

There are 1 best solutions below

0
On

Once you’ve got a basis for the left eigenspace of $1$ (i.e., the null space of $P^T-I$), normalize the basis vectors individually to get the basis $\{\mathbf\pi_i\}$. Then, any convex combination $\sum_i c_i\mathbf\pi_i$, with $\sum_i c_i=1$ and $c_i\ge0$ will be a valid stochastic vector that is a stationary distribution of the chain.

In this case, we can immediately find the eigenvectors $(0,0,1,0,0)$ and $(0,0,0,1,0)$ by inspection, and observing that summing the first, second and last rows gives the third independent eigenvector $(1,1,0,0,1)$. Normalized, this becomes $\frac13(1,1,0,0,1)$, and so all of the stationary distributions have the form $$\frac {c_1}3(1,1,0,0,1)+c_2(0,0,1,0,0)+c_3(0,0,0,1,0) = \left(\frac{c_1}3,\frac{c_1}3,c_2,c_3,\frac{c_1}3\right).$$ Setting $c_1=3a$, $c_2=b$ and $c_3=c$ produces the given answer.