how to determine transient and recurrent state from transition matrix

Question

I wonder how can I determine the transient and recurrent state from transition matrix ? I mean if I have 10 states It would be very hard to draw diagram for them so how to analyse the matrix? For example consider : $$ \begin{bmatrix} 0.1 & 0.2 & 0.7 & 0 \\ 0.7 & 0 & 0.2 & 0.1 \\ 0.6 & 0.4 & 0 & 0\\ 0 & 0.5 & 0.5 & 0 \end{bmatrix} $$ Can anyone tell me how to approach this problem I only want to know how to look at transition matrix and identify transient and recurrent states?

Thanks.

Answer 1

The formal way to do this and as defined in the book Introduction to Probability Models by Sheldon Ross is:

A state $i$ is recurrent if $\sum_{n=1}^{\infty}p_{ii}^n = \infty$

A state $i$ is transient if $\sum_{n=1}^{\infty}p_{ii}^n < \infty$

You can also define this as:

A state $i$ where $i \in S$ is said to be recurrent if $f_i$ = $P$(ever returns to $i$| starts in $i$) = 1

A state $j$ where $j \in S$ is said to be recurrent if $f_j$ = $P$(ever returns to $j$| starts in $j$) < 1

You would need to do this for all states in your matrix and solve it.