Markov Chain - Absorption

Question

I am interested in learning about Markov chains, for that I am doing the following exercise and I am generating the following questions:

I have the following matrix of one-step transition probabilities:

''' \begin{equation} \begin{pmatrix} 0.6 & 0.4 & 0 & 0 & 0 & 0 & 0 & 0\\ 0 & 0.2 & 0.6 & 0 & 0.2 & 0 & 0 &0 \\ 0.2 & 0 & 0.3 & 0 & 0 & 0 & 0 &0.5 \\ 0 & 0 & 0 & 0 & 0 & 0.9 & 0 &0.1 \\ 0 & 0 & 0 & 0 & 0 & 1 & 0 &0 \\ 0 & 0 & 0 & 0 & 0.5 & 0 & 0.5 &0 \\ 0 & 0 & 0 & 0 & 1 & 0 & 0 &0 \\ 0 & 0 & 0 & 0 & 0 & 0 & 0 &1 \\ \end{pmatrix} \end{equation}

I will assume that the initial state is 2 with probability 0.4 and 4 with probability 0.6

I would like to know the following: A. The average time the chain visits state 1 before being absorbed B. The total average number of steps before being absorbed

Problems encountered while developing the problem:

1. When doing the problem, I would like to order the transient and absorbentes states to be able to separate the matrix of transition probabilities into sub-matrices, however, when I exchanging some rows and some columns, I don't understand how the new states work. I don't know which are absorbent and which are transient

2. For point A, I would like to find the mean time of the chain before being absorbed like this: the inversion of the following matrix (Identity Matrix - Transient States Matrix). to state 1 "

3. I don't know how to express point B in a matrix

thanks for your help

'''

Answer 1

The Wkipedia article on absorbing Markov chains tells you what you need to know.

First of all, it's easy to tell which states are absorbing. They are the states where there's a $1$ on the diagonal of the transition matrix. That means that once the system gets to that state, it stays there, which is what "absorbing" means. In your example, only the last state is absorbing, so you don't need to reorder the matrix.

As explained on the wiki, and as I gather you know, from your question, the key is to compute the fundamental matrix $$N=(I-Q)^{-1}$$ Then the $(,ij)$ entry of $N$ is the expected number of times that the chain visits state $j$, given that it stars in state $i$. The expected number of steps until absorption, given that the chain started in state $i$, is the $i$th entry of the vector $$N\mathbf{1},$$ where $N\mathbf{1}$ is a vector of all $1$'s. To put it another way, it's the sum of the entries in the $i$th row of $N$, that is, the expected number of visits to any absorbing state.