Markov Chains expected number of steps

113 Views Asked by At

I know that, if all the states are transient, we can get the expected number of steps between state i and state j by simply transforming state j into an absorbing state and do the standard process, where you go find the inverse of a certain matrix and add all the elements of the relevant line.

In the described algorithm you can tell the expected number of steps until the process, starting at the state i, is absorbed.

And so my question is, how can you do this when the original markov chain already has absorbing states besides the state j? How can you know which of the states ends up absorbing the process and thus determine the expected number of steps between state i and state j?