markov chain: how to find the probability of being in a state after n steps?

306 Views Asked by At

Assuming we start at state 0:

enter image description here

enter image description here

Attached is a picture of the transition matrix and the problem I am trying to figure out. I understand that we have to go from 0 to 7, giving that a probability of .5, and I understand the n-1 being the steps from 7 to itself. However, I'm having a hard time understanding where the 0.4 comes from.

Thanks for your help!

1

There are 1 best solutions below

4
On BEST ANSWER

This is a very sloppily drawn diagram because they do not label the self-loops explicitly. The point is that there is only 0.6 worth of outgoing edges from state 7, which means that by conservation of probability there must be a self-loop with weight 0.4 from state 7 to itself.