Clarification on Periodicity of Markov Chains

Question

Given the Markov Chain: \begin{bmatrix} 0 & 1 & 0 & 0 & 0 & 0\\ 0 & 0 & 0 & 1 & 0 & 0\\ 0.3 & 0 & 0 & 0.7 & 0 & 0\\ 0 & 1 & 0 & 0 & 0 & 0\\ 0 & 0 & 0 & 0.4 & 0 & 0.6\\ 0 & 0 & 0 & 0 & 1 & 0 \end{bmatrix} The question asks to find the periodicity of each class, and I figured out all the states except state 1 and 3. I believe the period of both of them are 0 i.e. no period since we can't return to either 1 or 3 if we start at these states. However, since all states communicate with itself, does it mean the period of 1 and 3 is 1?

Answer 1

As you noted, if the Markov chain starts either in state 1 or state 3 it will never comeback to that state. Such states are called non-return states. Every state communicates with itself in the sense of zero-step transition probabilities, defined by \begin{equation*} p_{ij}^{(0)}=\left\{\begin{array}{ccc} 1 & if & i=j \\ 0 & if & i\ne j \\ \end{array}\right. \end{equation*} However, periodicity of a state $j$ is defined as the gcd of a non-empty set of all those $n>0$, for which $p_{jj}^{(n)}>0$. Period of a state is not defined otherwise. Clearly, period is not defined for states other than 2 and 4, in the given transition probability matrix. States 2 and 4 form a closed-communicating class with periodicity 2.

Answer 2

In your case, for all $n>0$, $p^{(n)}_{11} = p^{(n)}_{33}=0$ thus, according to Wikipedia, the period is undefined (or $+\infty$ if you want, it's a matter of convention).