If I have a n $\times$ n transition matrix and want to find the expected times passing the point j if the starting point is i and the amount of steps is m.
For example $$P =\begin{bmatrix}0.5&0.5&0&0&0\\0.5&0&0.5&0&0\\0.5&0&0&0.5&0\\0&0&0.5&0&0.5\\0&0.5&0.5&0&0\end{bmatrix}$$
i = 1, j = 5, m = 100
Thank you.
The transition probabilities after $k$ steps become $\mathbf{P}^k$. Hence, the $(i, j)$ entry of matrix $\mathbf{N}$ below is the expected number of times the chain is in state $j$, given that the chain started in state $i$.
$$\mathbf{N}=\sum_{k=0}^m{\mathbf{P}^k}$$
If the matrix $\mathbf{I-P}$ is invertible, then the above sum can be expressed as $(\mathbf{I}-\mathbf{P}^{m+1})(\mathbf{I}-\mathbf{P})^{-1}$ using the geometric sum formula.
Otherwise, if $\mathbf{P}$ is diagonalizable $$\mathbf{P}=\mathbf{Q}\mathbf{\Lambda}\mathbf{Q}^{-1}$$ where $\mathbf{\Lambda}$ is a diagnoal matrix with elements $\lambda_i$ that are eigen values of $\mathbf{P}$. The matrix $\mathbf{N}$ is then can be given by $$\mathbf{N}=\mathbf{Q}\mathbf{D}\mathbf{Q}^{-1}$$ where $\mathbf{D}$ is a diagonal matrix whose diagonal elements are given by $$d_i=\begin{cases}m+1,&\lambda_i=1\\\frac{1-\lambda_i^{m+1}}{1-\lambda_i},&\lambda_i\neq1\end{cases}$$