Markov chain - probability of being in state 4 after five steps?

97 Views Asked by At

enter image description here

I need to find the probability of being in state 4 after five steps in this markov chain. My first inclination would be that it has to go to 7 then 4, so

(.6)(.5) then subtract probability of leaving 4 (1-.1); I would then put it to the 3rd power because there are 5 steps - 2 taken to go to 7 and 4.

so: (.6)(.5)(1-.1)^3

Where am I going wrong?

1

There are 1 best solutions below

1
On BEST ANSWER

From your remarks, I'm assuming you're starting at $0$. There's also the possibility of stalling at $7$ on the way to $4$.

$$p(0\to7\to7\to7\to7\to4) + p(0\to7\to7\to7\to4\to4) + p(0\to7\to7\to4\to4\to4)+ p(0\to7\to4\to4\to4\to4)\\ = (0.6 \times 0.4^3 \times 0.5) + (0.6 \times 0.4^2\times 0.5\times 0.9) + (0.6\times0.4\times 0.5\times0.9^2) + (0.6\times0.5\times0.9^3)$$

You were only considering the last of these.