Aperiodicity of Markov chain

3.3k Views Asked by At

If a markov chain which has many states but only one state has a self-loop edge, then does it mean that the markov chain is aperiodic? Or every state in the markov chain has to have self-loop? For instance, in the following markov-chain, only state 2 has a self-loop edge, would this edge make the markov-chain aperiodic? :

Example of markov-chain

3

There are 3 best solutions below

0
On BEST ANSWER

Yes, the Markov chain you gave is aperiodic. To see this, you can try proving that whenever the underlying graph is strongly connected (or, in other words, the MC is irreducible) and contains at least one self-loop, then the Markov chain is aperiodic.

2
On

The definition of aperiodic is that every state is aperiodic, meaning that $\gcd\{t: \ P(X_t=s | X_0=s)>0\}=1$ for all states $s$. You can easily verify this to be the case for your Markov chain with one state and a self-loop.

0
On

The Markov chain is irreducible, aperiodic. To see this, note $ 1 \leadsto 3 $ as $ p_{1,3} = 0.4 > 0 $, $ 3 \leadsto 2 $ as $ p_{3,2} = 0.1 > 0 $, $ 2 \leadsto 4 $ as $ p_{2,4} = 0.87 > 0 $, $ 4 \leadsto 5 $ as $ p_{4,5} = 0.75 > 0 $, $ 5 \leadsto 6 $ as $ p_{5,6} = 0.5 > 0 $, $ 6 \leadsto 7 $ as $ p_{6,7} = 0.3 > 0 $ and finally $ 7 \leadsto 1 $ as $ p_{7,1} = 1 > 0 $.

Since periodicity is a class property, enough to show any state has period $1$. Note $ p_{2,2} = 0.1 > 0 $, so the period of $ 2 $ must be $1$.