Expected Time Until Absorption for Discrete Finite State Markov Chain

109 Views Asked by At

The Problem:

Consider a discrete time Markov chain with states $0,1,...,N$ whose matrix has elements $$ P_{ij} = \left\{ \begin{array}{ll} \mu_i, & j = i - 1; \\ \lambda_i, & j = i + 1, \hspace{5mm} i,j = 0,1,...,N; \\ 1 - \lambda_i - \mu_i, & j = i; \\ 0 & |j - i| > 1. \\ \end{array} \right. $$ Suppose that $\mu_0 = \lambda_0 = \mu_N = 0$, and all other $\mu_i$'s and $\lambda_i$'s are positive, and that the initial state of the process is $k$. Determine the expected time until absorption.

My Progress:

So, I know that if we define $\rho_0 = 1$ and $$ \rho_i = \frac{\mu_1 \mu_2 \cdots \mu_i}{\lambda_1 \lambda_2 \cdots \lambda_i}, $$ then $$ \Pr\{\text{absorption at $0$} \} = 1 - \Pr\{\text{absorption at $N$} \} = \frac{\sum_{i=k}^{N-1}\rho_i}{\sum_{i=0}^{N-1}\rho_i} . $$

However, my thinking is as follows: If we collect the transient states into the set $T = \{1, 2, ..., N-1\}$, then the expected time until absorption is simply the mean time the process spends in $T$. Now, I know that, for $i,j \in T$, if $s_{ij}$ denotes the number of time periods that the process is in $j$ given that it starts in state $i$, we have the following formula: $$ s_{ij} = \delta_{ij} + \sum_{k=1}^{N-1}P_{ik}s_{kj}. $$ But I assume that I am supposed to use my knowledge of the above probabilities of absorption at $0$ or $N$ (perhaps in some kind of conditioning argument) to determine the desired time. Whatever the case, both approaches have sent me down different rabbit holes, neither of which has proven very fruitful.