Transition intensities of Markov process

Question

In Markov process, transition intensities from state i to j are defined as derivatives of transition probabilities at zero: $$q_{ij}=p_{ij}'(0)$$ However I can't somehow catch the interpretation of transition intensities. My first thought was that they stand for probability of immediate transition, but $q_{ii}$ can be negative, so it can't be true. Any tips/ideas would be appreciated.

Answer 1

The formula you have written is correct. The meaning might be clearer if you restate the situation in "Taylor expansion form" as:

$$P(X_{t+h}=j \mid X_t=i)=\begin{cases} q_{ij}h+o(h) & i \neq j \\ \left ( 1-\sum_{j=1,j \neq i}^n q_{ij} h \right ) + o(h), & i=j \end{cases}$$

as $h \to 0^+$. In particular, the derivatives of the diagonal entries of the transition matrix are indeed negative initially, because the probability that you stay at $i$ gradually decreases as time advances.

If this doesn't make sense, it may help to consider an equivalent interpretation, which decomposes a continuous time Markov chain into a "jump chain" and "holding times". More specifically, the jump chain is a discrete time Markov chain which says where the continuous time chain goes when it eventually makes its transition from a given state. The holding times are exponentially distributed random variables that describe how long it takes for the continuous time process to escape a state.

Being explicit, the jump chain probabilities are $r_{ij}=-\frac{q_{ij}}{q_{ii}}$ (or zero, if $q_{ii}=0$), while the holding times at each state $i$ have mean $-\frac{1}{q_{ii}}$ (or $+\infty$, if $q_{ii}=0$). Thus $-q_{ii}$ describes the average rate of escape from $i$ while $q_{ij}$, in relation to $q_{ii}$, describes how often the eventual escape from $i$ leads to $j$.

One surprising fact about this is that one can compute the sequence of states visited by a realization of a continuous time homogeneous Markov chain without actually computing when they were visited. This is quite useful in simulations.