Need a help with Markov Chain

99 Views Asked by At

We have Markov Chain with continuous time, three conditiions and generator: \begin{equation*} Q = \begin{bmatrix} -3 & 1 & 2\\ 1 & -1 & 0\\ 1 & 0 & -1 \end{bmatrix} \end{equation*}

Initial distribution $X_0=1$ almost sure. How to find the distribution of the second moment of the chain jump?

1

There are 1 best solutions below

1
On

Recall the Kolmogorov forward and backward equations which govern the transient behavior of the process. In particular, we have $$ P'(t) = QP(t), $$ where $P_{ij}(t) = (\mathbb P(X(t) = j\mid X(0)=i)$. The solution to this differential equation is the matrix exponential $$P(t) = \sum_{n=0}^\infty \frac{(tQ)^n}{n!}. $$ In this case, one can use diagonalization to compute powers of $Q$, and we find that $$ P(t) = \left( \begin{array}{ccc} \frac{3 e^{-4 t}}{4}+\frac{1}{4} & \frac{1}{4}-\frac{e^{-4 t}}{4} & \frac{1}{2}-\frac{e^{-4 t}}{2} \\ \frac{1}{4}-\frac{e^{-4 t}}{4} & \frac{1}{12} \left(e^{-4 t}+8 e^{-t}+3\right) & \frac{1}{6} \left(e^{-4 t}-4 e^{-t}+3\right) \\ \frac{1}{4}-\frac{e^{-4 t}}{4} & \frac{1}{12} \left(e^{-4 t}-4 e^{-t}+3\right) & \frac{1}{6} \left(e^{-4 t}+2 e^{-t}+3\right) \\ \end{array} \right). $$ Since $\mathbb P(X(0)=1)=1$, the distribution of the first jump is given by the minimum of two exponentially distributed random variables - namely, it is exponential with rate $3$. To analyze the distribution of the second jump, we must condition on the first jump. Let $\{J_n:n=1,2,\ldots\}$ denote the jump chain. Then $J_2$ is the sum of $J_1$ and a hyperexponential distribution: \begin{align} J_2 = J_1 + Y \end{align} where $$ f_Y(t) = \frac13 e^{-t}+\frac23 e^{-t} = e^{-t}, $$ which in this case happens to be exponential as well. We compute the density of $J_2$ by convolution: \begin{align} f_{J_2}(t) &= f_{J_1}\star f_Y(t)\\ &= \int_0^t f_{J_1}(\tau) f_Y(t-\tau)\ \mathsf d\tau\\ &= \int_0^t 3e^{-3\tau} e^{-(t-\tau)}\ \mathsf d\tau\\ &= \frac32 e^{-3t}(e^{2t}-1). \end{align}