First Step Analysis of a Markov Chain process

Question

I have a Markov Chain transition probability matrix as the following. The possible states are $\{0,1,2\}$ $$ P = \begin{bmatrix} 0.6 & 0.2 & 0.2 \\ 0.2 & 0.5 & 0.3 \\ 0 & 0 & 1 \end{bmatrix} $$

The question asks me the last non-absorbing state is $0$, starting from state $X_{0} = 0$.

I attempt the following: I let $T = \min\{n \geq 0, X_{n} = 2 \}$ be time of absorption and consider $u_{i} = P(X_{T-1} = 0 | X_{0} = i)$ I set up the equation of using First Step Analysis:

$$ u_{0} = 0.6 u_0 + 0.2u_1 + 0.2u_2 $$

$$ u_{1} = 0.2 u_0 + 0.5 u_1 + 0.3 u_2 $$

$$ u_2 = 1 $$

I solve this and get $u_1 = 44/50$ and $u_0 = 47/50$. But I checked the answer in the back it is said that $u_0 = u_1 = 1$. Does I misunderstand the statement to express in terms of $u_{i}$?

Answer 1

I wrote this $\texttt R$ code to simulate the process:

rm(list=ls())

N <- 10000

X <- rep(0, N)

for(i in 1:N) {
  state <- 0

  while(state!=2) {
    U <- runif(1)
    if(state==0) {
      if(U <0.6)
        state <- 0
      else if(U <0.8)
        state <- 1
      else {
        X[i] <- 0
        state <- 2
      }
    }
    else {
      if(U<0.2)
        state <- 0
      else if(U < 0.7)
        state <- 1
      else {
        X[i] <- 1
        state <- 2
      }
    }
  }
}

cat(sprintf("P(last non-absorbing state=0) = %f\n", length(which(X==0))/N))

This gives an approximate probability of $0.62$ that the last non-absorbing state is $0$. I hope it is useful.