If $P$ is a transition matrix, and $m_{ij}$ the mean return time, how to show $m_{ij} = 1+ \sum_{k \neq j}P_{ik}m_{kj}$?

Question

If $P$ is a transition probability matrix of a finite state regular Markov Chain, and $m_{ij}$ is the mean return time, how can I show that $m_{ij} = 1+ \sum_{k \neq j}P_{ik}m_{kj}$? It seems rather trivial but I don't know how to get the $k \neq j$ part. Does anyone have any ideas?

Answer 1

This equation comes from "first step analysis". The "1" counts the first step, while $\sum_{k \neq j}P_{ik}m_{kj}$ gives the number of additional steps needed, on average, to reach state $j$. In the case $k=j$, the chain jumps directly from $i$ to $j$, and zero additional steps are required. So the $k=j$ term is omitted.