Upgrading item problem - using Markov chain

Question

I have an item, let's say sword.

As described above,
If my sword is at state 1, I can try upgrading it. There are 3 possibilities.
It can be upgraded with prob = 0.3, remain still with prob = 0.68, can be destroyed with prob = 0.02.

If my sword is at state 2, I still can try to upgrade it.
It can be upgraded with prob = 0.3, can be downgraded to state 1 with prob = 0.68, can be destroyed with prob = 0.02.

Once my sword destroyed, there is no turning back.
Once my sword reached at state 3, no need to do something else. I'm done.

I know it's a Markov chain problem.
I can express this situation with matrix, and if I multiply it over and over, it can reach equilibrium state.

p2 = matrix(c(1, rep(0, 3),
              0.02, 0.68, 0.3, 0,
              0.02, 0.68, 0, 0.3,
              rep(0, 3), 1), 4, byrow = T)
p2
##      [,1] [,2] [,3] [,4]
## [1,] 1.00 0.00  0.0  0.0
## [2,] 0.02 0.68  0.3  0.0
## [3,] 0.02 0.68  0.0  0.3
## [4,] 0.00 0.00  0.0  1.0

matrix.power <- function(A, n) {   # For matrix multiplication
  e <- eigen(A)
  M <- e$vectors
  d <- e$values
  return(M %*% diag(d^n) %*% solve(M))
}

round(matrix.power(p2, 1000), 3)
##       [,1] [,2] [,3]  [,4]
## [1,] 1.000    0    0 0.000
## [2,] 0.224    0    0 0.776
## [3,] 0.172    0    0 0.828
## [4,] 0.000    0    0 1.000

But how can I get the Pr(Reach state 3 without destroyed | currently at state 2) using Markov chain?

I could get Pr(Reach state 2 without destroyed | currently at state 1) by using sum of geometric series.

Thank you.

Answer 1

You could try to compute the limit of $P^n$ as $n\to\infty$ and then read all four absorption probabilities from this matrix, but as BGM suggested in a comment, computing the fundamental matrix of this absorbing Markov chain is much simpler and will give you more information to boot.

We have, in canonical form, $$P=\begin{bmatrix}0.68 & 0.30 & 0 & 0.02 \\ 0.68 & 0 & 0.30 & 0.02 \\ 0&0&1 &0 \\ 0&0&0&1 \end{bmatrix}.$$ The fundamental matrix is then, to three decimal places, $$N = \begin{bmatrix} 1-0.68 & -0.30 \\ -0.68 & 1 \end{bmatrix}^{-1} = \begin{bmatrix} 8.621 & 2.586 \\ 5.862 & 2.759 \end{bmatrix}.$$ The row sums of this matrix are the expected number of steps before the process terminates, starting from state $1$ and $2$, respectively. So, for instance, starting from the base state, on average you’ll either max out the sword or break it after about $11$ upgrade attempts.

The absorption probabilities are $$NR = \begin{bmatrix} 8.621 & 2.586 \\ 5.862 & 2.759 \end{bmatrix} \begin{bmatrix} 0 & 0.02 \\ 0.30 & 0.02 \end{bmatrix} = \begin{bmatrix} 0.78 & 0.22 \\ 0.83 & 0.17 \end{bmatrix}.$$ The probabilities of eventually reaching state $3$ are the first column of this matrix, while the second column gives the probabilites of breaking the sword before reaching state $3$.

See this Wikipedia article or any other standard reference on absorbing Markov chains for details.

Answer 2

If $\ p_i=P(\text{reaching state $3$}| \text{currently in state $i$})\ $, then obviously $\ p_0=0, p_3=1\ $, and you can obtain $\ p_1, p_2\ $ by solving the pair of linear equations \begin{align} p_1&=0.3p_2+0.68p_1\\ p_2&=0.68p_1+0.3\ , \end{align} which give $\ p_1=\frac{45}{58}, p_2=\frac{24}{29}\ $.