Sojourn times and Kolmogorov equations

Question

I'm learning about continuous Markov processes (so the time is continuous and we have a discrete state space, I think these are also called jump processes), they are well-studied generalizations of the Poisson process.

I'm trying to understand the proof of the Kolmogorov forward/backward equations. I'm using the book of Rozanov and he assumes (Th. 8.2) the transition probabilities satisfy $$\begin{aligned} 1-p_{ii}(\Delta t) &= \lambda_i \Delta t + o(\Delta t), \qquad i=1,2,\dots \\ p_{ij}(\Delta t) &= \lambda_{ij} \Delta t + o(\Delta t),\qquad j\neq i, \ i,j=1,2,\dots \end{aligned}$$ In that case I understand the proof of the differential equations, but it seems to me that these conditions on $p_{ij}(\Delta t)$ should not be assumptions, but theorems instead, so I'm asking about a proof of these facts (if a book contains the proof in detail then great).

Let's take the second equation concerning $p_{ij}(\Delta t)$. After searching in books, I understood the proof that if $\tau$ is the time of first jump, then under $\mathbb{P}^i$, $\tau$ and $X_{\tau}$ are independent, where $\mathbb{P}^i(A):=\mathbb{P}(A\mid X_0=i)$. Consequently if we denote $\pi_{ij} = \mathbb{P}(X_{\tau}=j\mid X_0=i)$, then for $j\neq i$, \begin{align*} p_{ij}(\Delta t) &= \mathbb{P}^i(X_{\Delta t}=j) = \mathbb{P}^i(X_{\Delta t} = j \text{ and } \tau \le \Delta t)\\ &\le \mathbb{P}^i(X_{\tau}=j \text{ and } \tau\le \Delta t) + \mathbb{P}^i(\text{at least two jumps by time } \Delta t) \end{align*} The first term is good : by independence it becomes $\pi_{ij}\mathbb{P}^i(\tau\le \Delta t) = \pi_{ij}(\lambda_i\Delta t+o(\Delta t))$, this I understand. But what about the second term ? why is it $o(\Delta t)$ ?

I saw a reference arguing that, if $\tau_i$ is the sojourn time in state $i$, then \begin{align*} &\mathbb{P}^i(\text{at least two jumps by time } \Delta t) \\ & \qquad = \sum_{k\neq i} \mathbb{P}^i(\text{at least two jumps by time } \Delta t\mid X_{\tau}=k)\pi_{ik}\\ & \qquad = \sum_{k\neq i} \mathbb{P}(\tau_i+\tau_k \le \Delta t)\pi_{ik} \le \sum_{k\neq i} \mathbb{P}(\tau_i\le \Delta t\text{ and }\tau_k \le \Delta t)\pi_{ik}\\ &\qquad = \sum_{k\neq i} o(\Delta t)\pi_{ik} = o(\Delta t) \end{align*} where in the last line he says it's because the sojourn times in $i$ and $k$ are independent, but he provides no proof for this... any proof for this or different argument ?

Thank you !

Answer 1

I made my searches and finally found good books on this, which I'm sharing here if anyone had the same question.

First of all, the above equations are not necessarily true for an arbitrary continuous-time Markov chain. They hold true however if you assume you have a regular jump process. This means you add two technical conditions : that sample paths are right-continuous and that there are finitely many jumps in bounded intervals. In this case you can prove the above equations

• either by making the above scheme rigorous. For this first prove that $\tau$ and $X_\tau$ are independent, then see Rozanov's other book ``Probability Theory, Random Processes and Mathematical Statistics'' p. 95 and fill its gaps,
• or prove using a very different argument that $p_{ij}(t)$ are differentiable directly. For this approach, see Markov Chains. Gibbs Fields, Monte Carlo Simulation, and Queues'' by Bremaud, orContinuous-Time Markov Chains. An Applications-Oriented Approach'' by Anderson.