We are given a generator matrix $Q$ (Q-matrix) for a continuous time Markov chain $(X_t)_t$. We want to calculate the probabilities of:
- returning to State 3 before State 1, while starting at State 3: $P(T_3 < T_1 | X_0=3)$
- returning to State 3 before State 1, while starting at State 2: $P(T_3 < T_1 | X_0=2)$
- The following expected time: $E(\min\{T_1, T_3\} | X_0 = 2)$
In the above, $T=\inf\{t>0; X_t=i\}$
Here is $Q$:
$\begin{pmatrix} -6 & 3 & 1 & 2\\ 3 & -8 & 3 & 2\\ 2 & 1 & -7 & 4\\ 1 & 1 & 3 & -5 \end{pmatrix}$
I would be so grateful for your help!
I know it's similar to the "Gambler's ruin" problem, which I can solve, but I'm not sure if this can be solved in a similar fashion. The problem is that here, we start in State 3 but then want to calculate the probability of returning back to 3 before hitting 1.