Stationary Distribution of CTMC given a rate matrix Q

317 Views Asked by At

enter image description hereenter image description here

For c, I understand that from state 1, the chain will either stay at 1 or go to state 2, since the chain is reducible. I am wondering if the stationary distribution of $\pi_2$ is $1/2$ or if the $\lim_{t \to \infty} P(X_t = 2 | X_0 = 1)$ does not exist.

For d, I see that starting in state 1, the chain can either go to state 4, which is absorbing, or get stuck in an oscillating loop between states 2 and 3. Is this the same as part c? Or would the limit not exist for this CTMC?

I would appreciate any help or guidance in the right direction to solve these. Thank you!

1

There are 1 best solutions below

0
On

For (c) we can just consider the generator matrix $$ Q = \begin{pmatrix}-1 & 1\\1&-1\end{pmatrix}. $$ By symmetry it is clear that $\lim_{t\to\infty} \mathbb P(X_t=2\mid X_0=1) = \frac12$.

For (d), let $\tau=\inf\{t>0:X_t\ne 1\}$. It is clear that $X_\tau$ is uniformly distributed over $\{2,4\}$. Conditioned on $\{X_\tau=2\}$ we need only consider $$ Q = \begin{pmatrix} -2 & 2\\ 1 & -1 \end{pmatrix}. $$ Here we can observe that $$\lim_{t\to\infty} \mathbb P(X_t=2, X_\tau = 2\mid X_0=1) = \frac23.$$ This can be verified by solving the equations $\pi Q=0$,$ \sum_i \pi_i=1$ to find the stationary distribution. It follows that $$ \lim_{t\to\infty} \mathbb P(X_t=2\mid X_0=1) = X_\tau = \mathbb P(X_t=2,X_\tau=2\mid X_0=1)\mathbb P(X_\tau=2) = \frac13. $$