I am trying to get an intuition for transience, recurrence and null recurrence. I constructed an example MC for myself represented by this graph below:
I'm thinking that all of the states are technically transient since if we go from 1 to 2, we will never see 1 and 3 again. However, if we go from 1 to 3 we will never see 1, 2 and 4 again.
Also, how does recurrence and null recurrence play into this? For example, if the nodes 2 and 4 didn't exist, then clearly 1 would be transient and 3 would be recurrent. But under these circumstances it's harder for me to determine which nodes are recurrent.
p.s. there should be a loop back to 3 with probability assignment 1.
Note that the following must hold for a recurrent state:
If $x$ is a recurrent state and $\rho_{xy}>0$, then $\rho_{yx}=1$. That is to say, if there is a nonzero probability that the state $x$ could go to another state $y$, then the probability that the state $y$ would in some time reach $x$ is 1.
Whenever we have closed, irreducible sets, the states contained in those sets are recurrent. For example, $\{3\}$ and $\{2,4\}$ are closed irreducible sets and these states are recurrent. You could check with the condition above.
In the meantime, there are some texts that are available online for stochastic processes:
Introduction to Probability Models by Sheldon Ross
Essentials of Stochastic Processes by Richard Durrett (I personally don't enjoy this book)