Positive recurrent, zero recurrent and transient states

Question

How do we prove that a state in a Markov chain process is positive recurrent, zero recurrent or transient? For example, if we have a transition matrix

$$P=\left(\begin{array}{ccc} \frac{1}{3} & \frac{1}{3} & \ 0 & \frac{1}{3} \\ 0 & \frac{1}{2} & \frac{1}{2} & 0\\ \frac{1}{2} & \ 0 & 0 & \frac{1}{2} \\ 0 & \frac{1}{3} & \frac{1}{3} & \frac{1}{3} \end{array}\right)$$

how do we do it??

Thanks a lot!!

Answer 1

If the state space is finite, start by dividing the state space into communicating classes.

Now check each class:

• If it is possible to leave the class, then all states in that class are transient.
• If it is not possible to leave the class, then all states in that class are positive recurrent.

There are no null recurrent states.

Answer 2

The path $1\to4\to2\to3\to1$ enumerates the state space and has positive probability hence every state is positive recurrent.