Irreducible Markov Chain Cycle can have repeating states?

372 Views Asked by At

If the PTM of an irreducible markov chain is given as follows where the states are $S = {1, 2, 3, 4}$

$\begin{bmatrix} .25 & .25 & .25 & .25 \\ 1 & 0 & 0 & 0 \\ 0 & 1 & 0 & 0 \\ 0 & 0 & 1 & 0 \end{bmatrix}$

would a sample cycle through all the states be $1,2,1,3,2,1,4,3,2,1$? I know irreducibility means that any pair of states is accessible. However, are repetitions of states allowed?

1

There are 1 best solutions below

0
On

Consider that the definition of irreducibility is that the chain, from state $i$, may visit an arbitrary state $j$ with positive probability.

$j$ here can be any given state in the Markov chain, including $i$, so if the chain is irreducible, repeats are allowed.

As an example, consider the case where we start in state 1.
$P(X_{n+1} = 1|X_{n} = 1) = 0.25$ as taken from the first element of the matrix. Alternatively, to go by the order you have given

$P(X_{n+1} = 2|X_{n} = 1) = 0.25$, and then $P(X_{n+1} = 1|X_{n} = 2) = 1$. Indeed, we can tell from your transition matrix that starting from any given state, we return to state 1 in an easily calculable finite time.