Find conditional probability of visiting a state in Continuous-Time-Markov-Chain

191 Views Asked by At

Given a state space {$1, 2, 3, 4$}, and the generator matrix:

$Q=\left(\begin{matrix} -2 & 1 & 1 & 0 \\ 1 & -3 & 1 & 1 \\ 2 & 2 & -4 & 0 \\ 1 & 2 & 3 &-6 \end{matrix} \right)$

I'm trying to find $P(X_1 = 3|X_0 = 1)$ and $P(X_5 = 1,X_2 = 4|X_1 = 3)$, but am clueless how to structure both in the continuous-time setting?

For the first part, I found the embedded MC:

$P=\left(\begin{matrix} 0 & 1/2 & 1/2 & 0 \\ 1/3 & 0 & 1/3 & 1/3 \\ 1/2 & 1/2 & 0 & 0 \\ 1/6 & 1/3 & 1/2 & 0 \end{matrix} \right)$

and $P(X_1 = 3|X_0 = 1)$ would correspond to multiplying by $\left(\begin{matrix} 1 & 0 & 0 & 0\end{matrix} \right)$, and taking the third column, i.e. $1/2$. Is this sound?