Markov chain period

Question

Let a Markov chain with State space $E=\{1,2,3,4\}$ and probability transition matrix:

$$P=\begin{bmatrix} 0 & 1 & 0 & 0 \\ 1/4 & 0 & 1/4 & 1/2\\0 & 1& 0 & 0 \\ 0 &1/2&0&1/2 \end{bmatrix}$$

How can I find the period of the chain?

---

I think the states 1,2,3 are periodic with period 2 and 4 is aperiodic because there is a loop.

Answer 1

Set $\DeclareMathOperator \gcd{gcd}$

$$N_x := \{n \in \mathbb{N}; p^n(x,x)>0\} \qquad \quad d_x := \gcd(N_x).$$

• state 4: Since $p(4,4) = \frac{1}{2}>0$, it follows that $p^n(4,4)>0$ for all $n \in \mathbb{N}$; hence, $N_4 = \mathbb{N}$ and $d_4 = \gcd(\mathbb{N})=1$, i.e. state $4$ is aperiodic.
• state 3: We have $$p^2(3,3) = \mathbb{P}^3(X_2 = 3) \geq \mathbb{P}^3(X_2 = 3, X_1 = 2) = 1 \cdot \frac{1}{4} >0,$$ i.e. $2 \in N_3$. Moreover, $$\begin{align*} p^5(3,3) &= \mathbb{P}^3(X_5 = 3) \\ &\geq \mathbb{P}^3(X_1 = 2, X_2 = 4, X_3 = 4, X_4 = 2, X_5 = 3) \\ &= 1 \cdot \frac{1}{2} \cdot \frac{1}{2} \cdot \frac{1}{2} \cdot \frac{1}{4} >0. \end{align*}$$ Consequently, $5 \in N_3$ and this implies $d_3 \leq \gcd(2,5)= 1$.

The argumentation for state 1 and 2 is similar, I leave it to you.