How many stationary distributions does the chain admit?

72 Views Asked by At

Consider the Markov chain whose transition matrix is

$$ P = \left( \begin{array}{ccc} 1 & 0 & 0 \\ \frac{1}{4} & \frac{1}{2} & \frac{1}{4} \\ 0 & 0 & 1 \\ \end{array} \right) $$

How many stationary distributions does the chain admit?

I did the $\pi P=\pi , \pi=(\pi_1,\pi_2,\pi_3)$ and found that $\pi_2=0$ so it doesn't admit a stationary distribution? And why?

1

There are 1 best solutions below

0
On

Your calculations are correct but the conclusion you draw from them is not. Your chain has an infinite family of stationary distributions, all of the form $(p,0,q)$ for arbitrary $p,q\in[0,1]$ for which $p+q=1$.

None of them, as you stated, put any mass on state $2$, which is a "transient state". The only way the chain can visit state $2$ is if it starts there. If it starts in state $2$ it will stay there for finitely many turns and then move to one of the other states, which are both "absorbing".