Probability and expected number of steps of/from a transient to an absorbing state in a Markov Chain

274 Views Asked by At

Given an arbitrary but relatively large (more than 50 states) transition matrix, the goal is to find the answer to the following problem:

Assuming we start at state A and ultimately end up in (the only) absorbing state Z we are looking for a state S that follows:

A → [m steps] → S → [n steps] → Z

How can I calculate the probability of reaching S before reaching Z and the corresponding expected number of steps (n) from S to Z?

Update 24-12:

So I guess the first part is mosre doable than i thought (if my reasoning is correct). My approach:

  1. Rearrange the matrix in canonical form:

    $P = \left[\begin{matrix} Q & R \\ \mathbf{0} & I \end{matrix}\right]$

  2. Find the fundamental matrix:

    $ N = (I - Q)^{-1} $

  3. Find the transient probabilities:

    $ H = (N - I)N_{dg} $

  4. And the row of H which corresponds to starting state A. The entries in this row vector should correspond to the probabilities of reaching A before reaching Z.

** Is my thinking here correct? And how can I condition the original transition matrix P to estimate the steps (n) between A and Z? **