Write a system of differential equation for the probabilities that the system is in state A, B, or C after time t.

Question

Consider the following process. A system is in one of the three states A, B, or C. A Poisson process is run with rate $\lambda$. At each event in this process, the following happens:

If the system is in state A, it moves to either B or C, chosen at random.

If it is system is in state B, it moves to state C.

If it is system is in state C, it moves to state A, B or C with probability 0.2, 0.3, or 0.5.

Write a system of differential equation for the probabilities that the system is in state A, B, or C after time t.

Can anyone suggest me how we think in this question. I never solve this type of question before.

Answer 1

The first thing to do is deal with the rate $\lambda$. This is just a constant that tells what is the number of transitions per unit time. I prefer to think of it as the constant that allows me to write dimensionless equations. So if $\frac {dP}{dt}$ has units of $s^{-1}$, and $\lambda$ has the same units, then $\frac {dP}{\lambda dt}$ is dimensionless.

If the only transitions are described by the first equation, the change in probability for the $A$ state would be $$\frac{dP_A}{\lambda dt}=-P_A$$ This means that everything that was $A$ should be gone. These would transform into $B$ or $C$ state, with equal probability. This can be written as:$$\frac{dP_B}{\lambda dt}=0.5 P_A\\\frac{dP_C}{\lambda dt}=0.5 P_A$$

Now consider the effect of the transition from $B$ to $C$. That transition says that $P_B$ will decrease, and the rate is $-P_B$. Meanwhile $P_C$ will increase at the rate $P_B$. So now the three equations can be written as $$\begin{align}\frac{dP_A}{\lambda dt}&=-P_A\\ \frac{dP_B}{\lambda dt}&=0.5 P_A-P_B\\\frac{dP_C}{\lambda dt}&=0.5 P_A+P_B\end{align}$$

The last is the transition from $C$. The rate of transition to $A$ will be $0.2P_C$, to $B$ will be $0.3P_C$. But you can also transition from $C$ to $C$. The rate for this will be $0.5P_C$. Note however that you need to decrease $P_C$ by all the transitions to the same or other states. We can therefore write the final equations as: $$\begin{align}\frac{dP_A}{\lambda dt}&=-P_A+0.2P_C\\ \frac{dP_B}{\lambda dt}&=0.5 P_A-P_B+0.3P_C\\\frac{dP_C}{\lambda dt}&=0.5 P_A+P_B+(0.5-1)P_C\end{align}$$

Answer 2

Let $X(t)$ be the state of the system at time $t$, then $\{X(t):t\geqslant0\}$ is a continuous-time Markov chain with infinitesimal generator $$ Q = \left( \begin{array}{ccc} -\lambda & \frac{\lambda }{2} & \frac{\lambda }{2} \\ 0 & -\lambda & \lambda \\ \frac{\lambda }{5} & \frac{3 \lambda }{10} & -\frac{\lambda }{2} \\ \end{array} \right). $$ We have the Kolmogorov backward equations $P'(t) = QP(t)$, which are \begin{align} P'_{AA}(t) &= -\lambda P_{AA}(t) + \frac\lambda 2P_{BA}(t) + \frac\lambda 2P_{CA}(t)\\ P'_{AB}(t) &= -\lambda P_{AB}(t) + \frac\lambda 2P_{BB}(t) + \frac\lambda 2P_{CB}(t)\\ P'_{AC}(t) &= -\lambda P_{AC}(t) + \frac\lambda 2P_{BC}(t) + \frac\lambda 2P_{CC}(t)\\ P'_{BA}(t) &= -\lambda P_{BA}(t) + \lambda P_{CA}(t)\\ P'_{BB}(t) &= -\lambda P_{BB}(t) + \lambda P_{CB}(t)\\ P'_{BC}(t) &= -\lambda P_{BC}(t) + \lambda P_{CC}(t)\\ P'_{CA}(t) &= \frac\lambda 5 P_{AA}(t) + \frac{3\lambda}{10} P_{BA}(t) -\frac\lambda 2P_{CA}(t)\\ P'_{CB}(t) &= \frac\lambda 5 P_{AB}(t) + \frac{3\lambda}{10} P_{BB}(t) -\frac\lambda 2P_{CB}(t)\\ P'_{CC}(t) &= \frac\lambda 5 P_{AC}(t) + \frac{3\lambda}{10} P_{BC}(t) -\frac\lambda 2P_{CC}(t). \end{align} It is readily verified that the solution to these equations is given by the matrix exponential $e^{tQ}$, which in this case does not have a particularly nice closed form: $$ \left( \begin{array}{ccc} \frac{1}{8} \left(e^{-\frac{1}{4} (5 t \lambda )} \left(7 \cos \left(\frac{1}{4} \sqrt{\frac{3}{5}} t \lambda \right)+\sin \left(\frac{1}{4} \sqrt{\frac{3}{5}} t \lambda \right) \sqrt{15}\right)+1\right) & \frac{1}{4} \left(e^{-\frac{1}{4} (5 t \lambda )} \left(\sqrt{15} \sin \left(\frac{1}{4} \sqrt{\frac{3}{5}} t \lambda \right)-\cos \left(\frac{1}{4} \sqrt{\frac{3}{5}} t \lambda \right)\right)+1\right) & \frac{5}{8}-\frac{1}{8} e^{-\frac{1}{4} (5 t \lambda )} \left(5 \cos \left(\frac{1}{4} \sqrt{\frac{3}{5}} t \lambda \right)+3 \sin \left(\frac{1}{4} \sqrt{\frac{3}{5}} t \lambda \right) \sqrt{15}\right) \\ \frac{1}{8}-\frac{1}{24} e^{-\frac{1}{4} (5 t \lambda )} \left(3 \cos \left(\frac{1}{4} \sqrt{\frac{3}{5}} t \lambda \right)+5 \sin \left(\frac{1}{4} \sqrt{\frac{3}{5}} t \lambda \right) \sqrt{15}\right) & \frac{1}{12} \left(e^{-\frac{1}{4} (5 t \lambda )} \left(9 \cos \left(\frac{1}{4} \sqrt{\frac{3}{5}} t \lambda \right)-\sqrt{15} \sin \left(\frac{1}{4} \sqrt{\frac{3}{5}} t \lambda \right)\right)+3\right) & \frac{1}{24} e^{-\frac{1}{4} (5 t \lambda )} \left(7 \sqrt{15} \sin \left(\frac{1}{4} \sqrt{\frac{3}{5}} t \lambda \right)-15 \cos \left(\frac{1}{4} \sqrt{\frac{3}{5}} t \lambda \right)\right)+\frac{5}{8} \\ \frac{1}{120} e^{-\frac{1}{4} (5 t \lambda )} \left(7 \sqrt{15} \sin \left(\frac{1}{4} \sqrt{\frac{3}{5}} t \lambda \right)-15 \cos \left(\frac{1}{4} \sqrt{\frac{3}{5}} t \lambda \right)\right)+\frac{1}{8} & \frac{1}{4}-\frac{1}{60} e^{-\frac{1}{4} (5 t \lambda )} \left(15 \cos \left(\frac{1}{4} \sqrt{\frac{3}{5}} t \lambda \right)+\sin \left(\frac{1}{4} \sqrt{\frac{3}{5}} t \lambda \right) \sqrt{15}\right) & \frac{5}{8}-\frac{1}{24} e^{-\frac{1}{4} (5 t \lambda )} \left(\sqrt{15} \sin \left(\frac{1}{4} \sqrt{\frac{3}{5}} t \lambda \right)-9 \cos \left(\frac{1}{4} \sqrt{\frac{3}{5}} t \lambda \right)\right) \\ \end{array} \right). $$ However, taking the limit as $t\to\infty$ yields $$ \lim_{t\to\infty} e^{tQ} = \left( \begin{array}{ccc} \frac{1}{8} & \frac{1}{4} & \frac{5}{8} \\ \frac{1}{8} & \frac{1}{4} & \frac{5}{8} \\ \frac{1}{8} & \frac{1}{4} & \frac{5}{8} \\ \end{array} \right), $$ meaning that independent of the initial state, we have $$ \lim_{t\to\infty} \mathbb P(X(t) = i) = \begin{cases} \frac18,& j=A\\ \frac14,& j=B\\ \frac58,& j=C. \end{cases} $$