Compute a stationary distribution for this chain. Is it unique? Compute $m_{i}=E_{i}(\tau_{i}(1))$ for each $i\in S$. Assume $S=\{1,2,3,4\}$.

28 Views Asked by At

Classify the states for the Markov chain with matrix $$P=\begin{pmatrix} 0.5 & 0 & 0.5 & 0\\ 0 & 0.2 & 0 & 0.8\\ 0.25 & 0 & 0.75 & 0\\ 0 & 0.6 & 0 & 0.4 \end{pmatrix}.$$

Compute a stationary distribution for this chain. Is it unique? Compute $m_{i}=E_{i}\left(\tau_{i}(1)\right)$ for each $i\in S$. Assume $S=\{1,2,3,4\}$.

This is what I get until now but I am having problems with the uniqueness and computing $m_{i}=E_{i}\left(\tau_{i}(1)\right)$.

\begin{align*} x_{1}&=0.5x_{1}+ 0x_{2} + 0.25x_{3}+0x_{4}\\ x_{2}&=0x_{1}+ 0.2x_{2} + 0x_{3}+0.6x_{4}\\ x_{3}&=0.5x_{1}+ 0x_{2} + 0.75x_{3}+0x_{4}\\ x_{4}&=0x_{1}+ 0.8x_{2} + 0x_{3}+0.4x_{4}\\ 1&=x_{1}+x_{2}+x_{3}+x_{4} \end{align*}

\begin{align*} x_1 &= \frac{1}{3}-\frac{7}{12}x_4\\ x_2 &= \frac{3}{4}x_4\\ x_3 &= \frac{2}{3}-\frac{7}{6}x_4\\ x_4 &= x_4 \end{align*}

Stationary distribution

$$x = \left[\frac{1}{3}-\frac{7}{12}x_4,\frac{3}{4}x_4, \frac{2}{3} - \frac{7}{6}x_4, x_4\right].$$