Probability calculations using a generator matrix

212 Views Asked by At

I have a continuous-time Markov Process $\{X(t)\}_{t\geq 0}$ with state space $S=\{1,2,3,4\}$ and generator matrix \begin{equation} G=\begin{pmatrix} -\frac{1}{2} & \frac{1}{6} & \frac{1}{12} & \frac{1}{4} \\ \frac{1}{6} & -\frac{1}{2} & \frac{1}{4} & \frac{1}{12} \\ \frac{1}{12} & \frac{1}{6} & -\frac{1}{2} & \frac{1}{4} \\ \frac{1}{12} & \frac{1}{6} & \frac{1}{4} & -\frac{1}{2} \end{pmatrix} \end{equation}

What is the probability that, once the process has entered state 2, it visits state 3 next, after it has spent more than two time units in state 2?

The way I interpret this question is that I must calculate the probability that the process goes from state 2 to state 3, and the holding time at state 2 for more than two time units.

I can easily compute $p_{2,3}=\frac{g_{2,3}}{-g_{2,2}}=\frac{\frac{1}{4}}{\frac{1}{2}}=\frac{1}{2}$.

Now I believe to understand that the holding time at a state is exponentially distributed, in this case with parameter $\lambda=\frac{1}{2}$. Since the question desires that I spent more than two time units in state 2, I rearrange the probability function to $1-\mathbb{P}(\text{staying in state 2 for 2 time units})$

Then using the cumulative distrubition function of an exponential distribution I get $(1-(1-e^{-\frac{1}{2}\cdot 2}))=1-(1-e^{-1})=e^{-1}$.

Combining these two parts then gives therefore, $\frac{1}{2}e^{-1}$.

Is this the correct process in determining the solution?