Absorbing Markov chain but the absorbing states don't absorb right away

239 Views Asked by At

What's the name of a Markov chain where there are states which absorb if they return to themselves twice in a row?

Say I have $5$ states, and state $5$ absorbs if it gets hit twice in a row.

For example, the sequence $1\rightarrow 5\rightarrow 3\rightarrow \cdots$ is possible but if we have $1\rightarrow 3\rightarrow 5\rightarrow 5 \rightarrow \cdots$ then all subsequent states are necessarily $5$.

I've seen some rough work with these where it seems that some sort of "exit matrix" is subtracted from the transition matrix in order to calculate average number of steps and whatnot but I can't find any information about these particular kinds of Markov chains as I don't know what they're called.

1

There are 1 best solutions below

4
On BEST ANSWER

This is not a Markov chain. In a Markov chain, by definition the probability of transitioning from $5$ to $3$ or to $5$ is independent of the earlier history of the process and is entirely determined by the current state $5$.

As suggested in a comment, to model this behaviour with a Markov chain you can introduce a separate absorbing state $5'$ and have $5$ transition to that instead of to itself.