Long run behavior of a absorbing markov chain

685 Views Asked by At

$$A=\begin{pmatrix}1&0&0&0\\0&1&0&0\\0.2&0&0.6&0.2\\0&0.2&0.2&0.6\end{pmatrix}.$$

In the above matrix how do I calculate the probability that in the long run the system will be in 1 of the absorbing states?In particular say the probability of being in the 1st absorbing state in the long run?Is it 0.5?

1

There are 1 best solutions below

0
On

The probability the Markov chain ends up in one of the absorbing states is $1$.

Let $p_i$ be the probability of absorption in state 1 when starting in state $i$

Note $p_2=0$ $p_1=1$

$$p_3 = 0.2p_1+0.6p_3+0.2p_4$$

why?!

write down the equation for $p_4$, then solve.

Let $q_i$ be the probability absorption in state 2, note that $q_i+p_i=1$ (why?)