For a semi-Markov chain, let $i,j\in S$ where $S$ is the space of states. Let also $P=(p_{ij})_{i,j\in S}$ be the matrix of transition probabilities. We define the following probabilities: $e_{i,j}(n)=P(\text{the chain enters state j at time n } |\text{ the chain entered state i at time 0})$
and
$q_{i,j}(n)=P(\text{the chain is at state j at time n }|\text{ the chain entered state i at time 0})$.
I believe that it is almost obvious that $e_{i,j}(n)=\sum_{k\in S}q_{i,k}(n-1)p_{kj}$, hence $E(n)=Q(n-1)\cdot P$, where $E(n)=(e_{i,j}(n))_{i,j\in S}, Q(n)=(q_{i,j}(n))_{i,j\in S}$. My intuition says that there is a recursive (with respect to $n$) formula for the probabilities $e_{i,j}(n)$ but I can't really formulate it. anyone has an idea?