Showing that a DTMC is positive recurrent

Question

I have a DTMC with probability transition matrix

P=\begin{bmatrix}0.25&0.75&0\\0.25&0.5&0.25\\0&0.25&0.75\end{bmatrix}

How will I show that this its all states are positive recurrent?

Thanks!

Answer 1

Two simple Lemmas might be useful.

Lemma. A MC with finite state space has at least one recurrent state.

Intuitively, this is because if all states were transient, the MC would end up being nowhere, which cannot be true since the MC has to be somewhere at each time $n$.

Lemma. Let $x, y$ be two states. If $x \to y$ and $x$ is recurrent, then $y$ is recurrent.

Intuitively, this is because $x$ being recurrent means that the MC will visit $x$ infinitely many times, and every time it visits $x$, there is a positive chance of going to $y$, hence $y$ must be visited infinitely many times as well.

Now note that:

• The MC has a finite state space.
• The MC is irreducible.

The first bullet point says that the MC must have a recurrent state, and the second bullet point says that all states must be recurrent.

Finally, for a MC with finite state space, recurrent states are positive recurrent.