Probabilities and rates in a CTMC

159 Views Asked by At

I have a CTMC with the following transition rates for the states $a,b,c$: $$Q= \begin{bmatrix} -2 & 3& 3\\ 2& -4& 0\\ 0& 1 & -3 \end{bmatrix}$$ where those are the exponential rates per hour.
Now, if I'm in state $c$, what is the probability that after 2 hours I'll be in state $a$?
I feel like the solution would use the formula $P(t+s)=e^{(t+s)Q}$, but I cannot seem to find a way to impose the condition that I'm starting from the $c$ state. (No actual results needed, just the theory behind it).

Bonus question: the problem said that the transition rate from state $a$ to state $b$ was 2, but there was also a process involved that could make the transition "restart" that was a Poisson process with rate 3. For example I'm building a sand castle with a rate of 2 and waves are destroying it with a rate of 3.
Shouldn't the actual transition rate be $2*\frac{2}{5}=\frac{4}{5}$?

1

There are 1 best solutions below

1
On BEST ANSWER

Recall the Kolmogorov backward equation $P'(t) = QP(t)$ with initial condition $P'(0)=Q$. This has the unique solution $P(t) = e^{Qt}$, where $$ e^{Qt} = \sum_{n=0}^\infty \frac{t^nQ^n}{n!}. $$ In this case we have $$ \left( \begin{array}{ccc} \frac{2 e^{-5 t}}{5}+\frac{3}{5} & \frac{1}{10} \left(e^{-5 t} \left(5 e^t-8\right)+3\right) & \frac{1}{10} e^{-5 t} \left(-5 e^t+e^{5 t}+4\right) \\ \frac{3}{5}-\frac{3 e^{-5 t}}{5} & \frac{1}{10} \left(e^{-5 t} \left(12-5 e^t\right)+3\right) & \frac{1}{10} e^{-5 t} \left(5 e^t+e^{5 t}-6\right) \\ \frac{3}{5}-\frac{3 e^{-5 t}}{5} & \frac{3}{10} e^{-5 t} \left(-5 e^t+e^{5 t}+4\right) & \frac{1}{10} e^{-5 t} \left(15 e^t+e^{5 t}-6\right) \\ \end{array} \right), $$ and hence the probability in question is $$ P(2)_{ca} = \frac35\left(1- e^{-10}\right). $$

(Note that I have transposed the matrix $Q$, as the generator marix for a continuous-time Markov chain is usually written so that the rows sum to zero, as opposed to the columns.)