Questions about Markov Chains

31 Views Asked by At

Suppose the system is in state $6$ at time$ = 0$. Can we give the limit $\infty$ of the probability that it will remain in state $6$? Otherwise, which state will the system be in at time $= 2n$?

enter image description here

The answer is: states $6$ and $8$, but I can't figure out how can we get this result?

1

There are 1 best solutions below

0
On

You can make a directed graph where the states of the Markov chain are the vertices and there is an arrow from $x$ to $y$ if the probability of going from $x$ to $y$ in one step is positive. If you do this, starting with state 6, you will get something like this:

Partial graph of Markov chain

You could fill it in with the rest of the states, but that's not necessary for your question. From the graph, we can see that if you are in the set {6,8}, you will jump to the set {5,7,9}, and vice versa. Thus, starting at 6, after an even number of steps, you will be in the set {6,8}.