Hi I am doing an exercise where I have to sketch a diagram from a Markov process and then only from that sketch argue if the matrix is regular or not. The process is a happening on a square where clockwise transitions have the probability of a and anti clockwise transitions has zero probability. Also no transitions occurs on the diagonal, only between neighboring sites. My sketch looked something like this:
a
2 ----->->---- 3
| |
a | |
^ v a
| |
1 ---<--<------4
a
Only by looking at this sketch how am I supposed to determine if the transition matrix is regular or not?
Hint:
As I pointed out in a comment, the diagram can represent a Markov chain only if $\ a=1\ $.
By definition, a Markov chain is regular if there is some power $\ P^n\ $ of its transition matrix $\ P\ $ whose entries are all strictly positive. This means that if the chain starts off in state $1$ (say) with probability $1$ at a time $\ t=0\ $, then at time $\ t=n\ $ it has a positive probability of being in any of its states, since the probability of its being in state $\ i\ $ at time $\ t=n\ $ is just the $\ i^\text{th}\ $ entry in the first row of $\ P^n\ $ (assuming you're using row-stochastic transition matrices).
If the Markov chain represented by your diagram starts off in state $\ 1\ $ with probabiliity $1$ at time $\ t=0\ $, what possible states can it be in at time $\ t=4k+j\ $ when $\ j=1,2,3\ $ or $\ 4\ $? Is there any time $\ t=n\ $ when it could have a positive probability of being in every one of its four states?