Are these transient or recurrent states in a Markov chain?

Question

I have the following transition matrix for a Markov chain with states $A, B, C, D, E$:

$$ \left| \begin{array}{ccc} 0 & 0 & \frac{1}{2} & \frac{1}{2} & 0 \\ \frac{1}{2} & 0 & \frac{1}{2} & 0 & 0 \\ \frac{1}{2} & 0 & 0 & \frac{1}{2} & 0 \\ 0 & \frac{1}{2} & 0 & 0 & \frac{1}{2} \\ 0 & 0 & 0 & 0 & 1 \end{array} \right| $$

State $E$ is an absorbing state, but I am wondering how to classify the other states. After drawing out the transition diagram, it seems that all the other states are transient states as eventually we will end up in state $E$. There are no states where we 'get stuck' alternating back and forward. Is this correct?

Answer 1

Throwing MATLAB at the problem, I find that all but one eigenvalue has magnitude less than 1, and this one eigenvalue has left eigenvector corresponding to the distribution which is only at $E$, so my analysis agrees with yours. Note that there is a pair of complex eigenvalues, which represent cycles (for example the cycle between $A$ and $C$), but because their magnitude is less than 1 these cycles die out in time.

Answer 2

I seem to recall a theorem, in which if an absorbing state is accessible from a communicating class $C$, then all states in $C$ are transient.

Answer 3

You are correct. The following is the state transition diagram associated with the given transition probability matrix. From the diagram, we observe that, there are two communicating classes:

• {E} which is closed and hence, state E is a recurrent state and

• {A,B,C,D}, which is a non-closed communicating class. In finite state Markov chains, states belonging to a non-closed communicating classes become transient.