for a reducible DTMC finding the stationary distribution

12 Views Asked by At

I got the following transition matrix

which has 4 states and 2 irreducilbe the state are [1,2,3,4]

                         .5  .5  0     0 
                   P=    .5  .5   0     0
                          .5  0   .25    .5
                            0  .25 .25   .5 

So there are two irreducible states [1,2] and [3,4] that communicate with each other.

But I am not sure how to find the stationary distribution. Would I do for example

would I set up a system of lienar equation such that $\pi P=\pi$ so for example $\pi_1=.5\pi_1+.5\pi_2+.5\pi_3+0\pi_4$ all the way up to $\pi_4$

$\pi_4=.\pi_3+.5\pi_4$

and then not sure how to find $P^{100} $

1

There are 1 best solutions below

0
On BEST ANSWER

There are no transitions from $\{1,2\}$ to $\{3,4\}$, so the stationary distribution has support only in $\{1,2\}$. Since the transition matrix restricted to $\{1,2\}$ is symmetric under exchange of $1$ and $2$, the stationary distribution is $\left(\frac12,\frac12,0,0\right)$.