Show transient states do not communicate.

Question

Given $$P= \begin{bmatrix} \frac{1}{3} & 0 & \frac{1}{3} & 0 & 0 & \frac{1}{3} \\ \frac{1}{2} & \frac{1}{4} & \frac{1}{4} & 0 & 0 & 0 \\ 0 & 0 & 0 & 0 & 1 & 0 \\ \frac{1}{4} & \frac{1}{4} & \frac{1}{4} & 0 & 0 & \frac{1}{4} \\ 0 & 0 & 1 & 0 & 0 & 0 \\ 0 & 0 & 0 & 0 & 0 & 1 \end{bmatrix} $$

So $\{5\}$ is a recurrent class because $5$ is absorbing.

Let $A = \{2, 4\}$. Then $P_A$ is stochastic, so $A$ is closed. Since $p_{24}, p_{42} > 0$, we know $2 \leftrightarrow 4$. Since closed, all states communicate and finitely many states, $A$ is a recurrent class.

Since $p_{02}, p_{12}, p_{32} > 0$ we see $0, 1, 3 \rightarrow 2$ but $0, 1, 3 \notin A$. Since $A$ is closed, states $0$, $1$, and $3$ are all transient.

We see that column for state $3$ is all zero, so $p_{i3}^{(n)} = 0$ for all $n$. Therefore $\{3\}$ is a transient class.

Here's my question. Are $0$ and $1$ together a class or are they separate classes? How do I show that?

Answer 1

Here is the state transition diagram for the given transition probability matrix.

As you have observed, there are two closed communicating classes, viz. {5} and {2,4}. Hence, all these states are recurrent. The state 1 itself forms a communicating class, but the class is not closed. Hence, state1 is a transient state. Similarly, the communicating class of state 0 contains only state 0 and it is non-closed. Therefore, state 0 is also a transient state. State 3 does not communicate even with itself. Such a state is called non-return state and is a transient state.