Finding the general stationary distribution of a Markov chain

249 Views Asked by At

Consider a Markov chain with state space $$S = \{1, 2, 3, 4, 5\}$$ and transition matrix $$P = \begin{pmatrix} 0 & \frac 1 4 & \frac 1 4 & \frac 1 4 & \frac 1 4\\ \frac 1 2 & 0 & \frac 1 2 & 0 & 0\\ \frac 1 2 & \frac 1 2 & 0 & 0 & 0\\ 0 & 0 & 0 & 1 & 0\\ 0 & 0 & 0 & 0 & 1 \end{pmatrix}.$$ Find the general form of the stationary distribution for this chain.

My working

A simple transition diagram will show that states $1, 2$ and $3$ are transient while states $4$ and $5$ are recurrent. Moreover, since we have a finite state space, we just need to find the stationary distributions concentrated on each of the positive recurrent communication classes, that is, $[4] = \{4\}$ and $[5] = \{5\}$. Clearly, the stationary distributions concentrated on $\{4\}$ and $\{5\}$ are simply $(0, 0, 0, 1, 0)$ and $(0, 0, 0, 0, 1)$ respectively. Thus, the general form of the stationary distribution for this chain is $$\pi = (0, 0, 0, \alpha, 1 - \alpha)\ \forall\ \alpha \in [0, 1].$$

As we have just covered Markov chains and this is my first time encountering such a problem, I would like to know if my answer (and logic) are correct. If any part of my working is incorrect, please point out where my mistakes are :)