First post here and trying to understand Markov chains/processes. Apologies if the notation used is non-standard but I hope it all makes sense.
Let's assume we have a time homogeneous Markov process with n-step Transition Matrix M(n) and a finite state space with three states {State 1, State 2, State 3}. State 3 is an absorbing state
$$M(1) = \begin{bmatrix}a_{1,1} & a_{1,2} & a_{1,3}\\b_{2,1} & b_{2,2} & b_{2,3}\\0 & 0 & 1\end{bmatrix}^1$$
Let's now assume an individual starts in state 1. I am trying to understand how to find the probability that an individual transitions to state 3 only at the final transition i.e., only transitions to state 3 at time n. One way I have understood it is that we take the difference between the (1,3) entry of the n-step matrix and the (1,3) entry of the n-1 step matrix:
$$M(n) = \begin{bmatrix}A1^n & A2^n & A3^n\\B1^n & B2^n & B3^n\\0 & 0 & 1\end{bmatrix}^n$$ $$M(n-1) = \begin{bmatrix}A1^{n-1} & A2^{n-1} & A3^{n-1}\\B1^{n-1} & B2^{n-1} & B3^{n-1}\\0 & 0 & 1\end{bmatrix}^{n-1}$$
So the probability of transitioning to state 3 only at time n is given by:
$$A3^{n} - A3^{n-1}$$
$$A3^{n} - A3^{n-1} = A1^{n-1} \times a_{1,3} + A2^{n-1} \times b_{2,3} + A3^{n-1} \times 1 - A3^{n-1}$$
$$A3^{n} - A3^{n-1} = A1^{n-1} \times a_{1,3} + A2^{n-1} \times b_{2,3}$$
Since the first term is the probability of reaching state 3 after n transitions, given that we start in state 1 (considering all possible paths to state 3 e.g., transitioning to state 3 before time n and staying in state 3; transitioning to state 3 only at time n; and inter-transitions between state 1 and state 2 then transitioning to state 3 at any time between time 2 and n) and the second term represents all possible paths to reach state 3 within (n-1) transitions then subtracting the second from the first yields the probability of transitioning only at time n...right? Please clarify if incorrect.
The second way I looked at it is by finding the probability of starting in state 1 and not transitioning to state 3 up to time (n-1) and then transitioning to state 3 after one additional transition to time n. We therefore need the product of these probabilities.
Since M is a transition matrix, the rows sum to 1. Therefore, the probability of NOT transitioning to state 3 up to time (n-1) is:
$$1-A3^{n-1} = A1^{n-1} + A2^{n-1}$$
The probability of then transitioning to state 3 in one step is:
$$a_{1,3}$$ if in state 1 at time n-1 and $$b_{2,3}$$ if in state 2 at time n-1.
$$A1^{n-1}\times a_{1,3} + A2^{n-1} \times b_{2,3} $$
The two approaches seem to yield the same result though I am not sure if this is correct. If anyone responding could also share specific literature that discusses this particular problem or other questions and their solutions - that are similar to this - I would really appreciate it.
The notation is hard to read. Here is the solution with more standard notation.
Let $d$ be the state distribution at time zero. If $M$ is the (one-step) transition matrix, then $d^{\intercal}M^{n}$ is the state distribution at time $n$. Let $x$ be an absorbing state and $$ n_{x}=\inf\left\{ n\geq0\colon X_{n}=x\right\} $$ be the first time the chain is in state $x$. It follows that $$ \mathbb{P}(n_{x}\leq n)=d^{\intercal}M^{n}e_{x} $$ where $e_{k}$ is the $k$-th standard basis vector. Therefore, for $n \geq 1$, \begin{multline*} \mathbb{P}(n_{x}=n) =\mathbb{P}(n_{x}\leq n)-\mathbb{P}(n_{x}\leq n-1) \\=d^{\intercal}M^{n}e_{x}-d^{\intercal}M^{n-1}e_{x} =d^{\intercal}\left(M^{n}-M^{n-1}\right)e_{x} =d^{\intercal}M^{n-1}\left(M-I\right)e_{x}. \end{multline*}