Specify the classes of the following Markov chains, and determine whether they are transient or recurrent

2k Views Asked by At

Specify the classes of the following Markov chains, and determine whether they are transient or recurrent: $$\mathbb{P}_1=\begin{Vmatrix}0 & 1/2 & 1/2\\ 1/2 & 0 & 1/2\\ 1/2 & 1/2 & 0\end{Vmatrix},\quad\quad \mathbb{P}_2=\begin{Vmatrix}0 & 0 & 0 & 1\\ 0 & 0 & 0 & 1\\ 1/2 & 1/2 & 0 & 0\\ 0 & 0 & 1 & 0\end{Vmatrix}$$ $$\mathbb{P}_3=\begin{Vmatrix}1/2&0&1/2&0&0\\1/4&1/2&1/4&0&0\\1/2&0&1/2&0&0\\0&0&0&1/2&1/2\\0&0&0&1/2&1/2\end{Vmatrix},\quad \mathbb{P}_4=\begin{Vmatrix}1/4&3/4&0&0&0\\1/2&1/2&0&0&0\\0&0&1&0&0\\0&0&1/3&2/3&0\\1&0&0&0&0\end{Vmatrix}$$

My Attempt

  • $\mathbb{P}_1$: Closed class: $\{0,1,2\}$. All states are recurrent.

  • $\mathbb{P}_2$: Closed classes: $\{0,1,2,3\}$. States 0 and 1 communicate with state 3, which communicates with state 2, which communicates with states 0 and 1 again. Thus, all states are recurrent.

  • $\mathbb{P}_3$: Closed classes: $\{0,2\},\{3,4\}$ because states 0 and 2 communicate with each other and states 3 and 4 communicate with each other. Open class: $\{1\}$ because state 1 communicates with 0 but 0 does not communicate with 1. State 1 communicates with 2 but 2 does not communicate with 1. States 0, 2, 3, and 4 are recurrent. State 1 is transient.

  • $\mathbb{P}_4$: Closed classes: $\{0,1\},\{2\}$ because states 0 and 1 communicate with each other, and state 2 communicates with itself. States 0 and 1 are recurrent, states 2, 3, and 4 are transient.

Can I get a formal and an intuitive definition of open and closed classes? I was able to figure this problem out from the patterns I have seen, but I would love to have a definition to go by.

I am not entirely sure if I did this correctly, so any help would be appreciated. Thanks.