As the title suggests, I am trying to understand how to compute a $n$-step transition probability given a transition probability matrix.
Please understand that this is purely for me to prepare for an exam and not a homework question (It might be too trivial to be a homework question anyway).
Suppose we have simple transition probability matrix, P denoted by:
\begin{bmatrix}
0 & 2/3 & 1/3 \\
1/2 & 0 & 1/2 \\
1/2 & 1/2 & 0 &
\end{bmatrix}
with three states, $X_{n}=\{0,1,2\}$. So, for example, $P_{01}$, which denotes the one step probability that a state at $0$ will be at state $1$ in the next step is $\frac{2}{3}$ as given in the matrix above.
My professor gave us a formula to calculate the $n$-step transition probability which is: $$P^{(n)}_{ij}=\sum_{l=0}^{\infty} P^{(m)}_{il}P^{(n-m)}_{lj}.$$
Suppose I want to calculate $P^{(3)}_{00}$ and say, $P^{(2)}_{23}$. How can I apply the formula to calculate the above probabilities?
I have also referred to: https://www.dartmouth.edu/~chance/teaching_aids/books_articles/probability_book/Chapter11.pdf
but it only had one numerical example of computing a 2-step transition probability. Can someone show me how to do it, step-by-step? Your help is much appreciated!