Simulate discrete state space CTMC from generator matrix

30 Views Asked by At

Consider a generator matrix $Q\in\mathbb{R}^{h\times h}$ for a discrete state space $\{1,...,h\}$. I want to determine the probability of a transition of a stochastic process $X(t)$ with $X(\bar{t})=i$ for $\bar{t}\in[0,\Delta t)$ and instantanuously $X(\Delta t)=j$ with both $i,j\in\{1,..,h\}$.

The transition probability matrix after $\Delta t$ is found by taking the matrix exponential of the generator matrix, thus $P_{\Delta t,i\rightarrow j}=\exp(Q\cdot \Delta t)_{i,j}$. The transition probability matrix takes into account every possible path from $X(0)=i$ to $X(\Delta t)=j$. However, I want to exclude this effect and therefore cannot use the transition probability matrix with timestep $\Delta t$.

In the derivation of the generator matrix and corresponding transition probability matrix we have several useful definitions. The stochastic process $X(t)$ with $X(0)=i$ remains in state $i$ for a random amount of time, which is exponentially distributed, thus $\sim\exp(-q_{i}\cdot\Delta t)$ with $q_{i}=-q_{ii}$. Given a transition from state $i$ at $\Delta t$ the conditional probability is defined as $\mathbb{P}(X(\Delta t)=j|X(\Delta t^{-})=i,X(\Delta t)\neq i)=\frac{q_{ij}}{q_{i}}$ with $q_{ij}=\lim_{\Delta t\rightarrow0}\frac{\mathbb{P}(X(\Delta t)=j|X(0)=i)}{\Delta t}$. .

My attempt was to simulate for every timestep $\Delta t$ the probability of a transition of $X(t)$ from state $i$ to $j$ by $\frac{1}{q_{i}}\exp(-q_{i}\cdot\Delta t)\frac{q_{ij}}{q_{i}}=q_{ij}\exp(-q_{i}\cdot\Delta t)$ with the probability of $X(t)$ remaining in state $i$ at $\Delta t$ to be $1-\sum_{j=1,j\neq i}^{h}q_{ij}\exp(-q_{i}\cdot\Delta t)$. For $\Delta t=1$ this approach seems to work; however, for $\Delta t\rightarrow 0$, there seems to be a factor $\Delta t$ missing. The following definition $q_{ij}\exp(-q_{i}\cdot\Delta t)\rightarrow q_{ij}\exp(-q_{i}\cdot\Delta t)\Delta t$ seems to do the job, but I cannot figure out where this factor $\Delta t$ is coming from.

Thanks in advance.