DTMC: Stationary Distribution with Recurrent Classes

384 Views Asked by At

I want to calculate the stationary probability, $\pi_j$ for a DTMC that contains two irreducible classes such as, $$ P_{ij} = \begin{pmatrix} 0 & 0 & 0 & 1 & 0 & 0\\ 0 & 0 & 0 & 1/2 & 1/2 & 0 \\ 1/2 & 0 & 0 & 0 & 0 & 1/2 \\ 1 & 0 & 0 & 0 & 0 & 0 & \\ 1/2 & 0 & 0 & 0 & 1/2 & 0 \\ 0 & 1/2 & 0 & 1/2 & 0 & 0 \\ \end{pmatrix} $$ (This should represent two irreducible recurrent classes $A\equiv\{0,3\}$ and $B\equiv\{1,4,5\}$: I just created this so let me know if I've made a typo)

I would like to calculate the long term behaviour of P,

$$ \rho = \lim_{n\to\infty}(P^n)_{ij} $$

Starting in $A$ or $B$, we are already in a periodic recurrent class. However, starting in $\{2\}$ we have an equal probability of transitioning to $\{0\}$ or $\{5\}$.

I am unsure how to incorporate this first step into my calculation of $\rho$

1

There are 1 best solutions below

0
On BEST ANSWER

$A=\{0,3\}$ is a closed class since $P_{03}=P_{30}=1$, and thus is also absorbing. It is also the only closed class. Since $$P_{13}=\frac12, P_{20}=\frac12, P_{40}=\frac12, P_{53}=\frac12$$ are positive, the probability of absorption is $1$. Since $$\sum_{j\in A} P_{ij} = \sum_{j\notin A} P_{ij} = \frac12$$ for each $i\notin A$, the time until absorption $\tau$ is geometrically distributed with parameter $\frac12$, and so $\mathbb E[\tau]=2$. The stationary distribution of $\{X_n\mid X_0\in A\}$ is simply $(\pi_0,\pi_3)=\left(\frac12,\frac12\right)$.