How to find the probability that a process never enters a particular state in a Markov Chain?

696 Views Asked by At

I am given the following Markov Chain:

Assume that the Markov Chain is in state 3 immediately before the first trial.

I am asked to find the probability that the process never enters state 1. I thought that the answer is $0.3$ (going to state 4), since if I go to state $2$ then I am sure to enter state 1 sometime (because {$1,2$} is a recurrent class). But that answer is wrong. The correct answer is $3/8$, but I don't see how they got to it. Any help will be appreciated.

2

There are 2 best solutions below

0
On BEST ANSWER

The chance $0.2$ arrow returns you to state $3$. All that matters to answer the question is whether you go to $2$ first or to $4$, because that determines whether you get to $1$. The chance of going to $4$ first is $\frac {0.3}{0.3+0.5}=\frac 38$

0
On

You also have to account for the possibility of going from state $3$ to state $3$ any number of times before going to state $4$. I.e., the allowable sequences are \begin{align*} &3 \to 4 \to \cdots \\ &3 \to 3 \to 4\to \cdots\\ &3 \to 3 \to 3 \to 4 \to \cdots\\ &\vdots \end{align*}