Getting Continuous Markov Chain Solution from the Discrete Solution

Question

If I have the transition probabilities matrix, $P_{ij}$, for the discrete case, it is possible to find the continuous case time-dependant transition probabilities matrix, $P_{ij}(t)$?

How to do this?

Answer 1

When you have a discrete time Markov Chain, you account with the information about where the chain jumps. This information is given by the transition probability matrix $P$.

On the other hand, if you have a continuous time Markov chain, you account with the information about where the chain jumps and when it jumps. This information is completely given by the infinitesimal matrix $Q$, defined by $$q_{ij}=\begin{cases}-\mu_i,&i=j,\\\mu_ip_{ij},&i\neq j, \end{cases}$$ where $\mu_1,\mu_2,...$ are the rates of the exponential distributions associated with the times of permanences in each estate.

So you can dirive the continuous tieme Markov chain only if you have the parameters $\mu_1,\mu_2,...$ (and the probabilities $p_{ij}$ given by $P$). Once you have the matrix $Q$, use the equation $P(t)=\exp\{tQ\}$.