Probability of a two-state continuous Markov chain

79 Views Asked by At

Consider a continuous-time Markov process ($\epsilon_t$) which takes two values ($\epsilon_t=0$ or $\epsilon_t=1$). Let $p_0$ denote the probability of switching from state 1 to 0 and let $p_1$ denote the likelihood of transitioning from state 0 to 1.

Can we calculate conditional probabilities like \begin{align} f(x|y)=P(\epsilon_t=x | \epsilon_0=y) \end{align} where $x,y\in\{0,1\}$?

1

There are 1 best solutions below

3
On BEST ANSWER

Yes, the calculation in the discrete time case is explained on page 3 of [1], and it works similarly in the continuous time case.

If $u_t=P(\epsilon_t=0 | \epsilon_0=0)$ then $$u_t'=p_0(1-u_t)-p_1u_t \,.$$ Thus $z_t=u_t-c$ satisfies $$z_t'=p_0(1-z_t-c)-p_1(z_t+c)=-(p_0+p_1)z_t+p_0-(p_0+p_1)c \,.$$ This suggests choosing $c=p_0/(p_0+p_1)$ so the equation for $z_t$ becomes $z_t'=-(p_0+p_1)z_t$. We have arrived at the most standard differential equation. By the product rule, $f(t)=e^{(p_0+p_1)t} z_t$ satisfies $f'(t)=0$ for all $t$, so $f(t)=f(0)=1-c$. we conclude that $$u_t=\frac{p_0}{p_0+p_1}+\frac{p_1}{p_0+p_1} e^{-(p_0+p_1)t} \,.$$

[1]https://yuvalperes.com/markov-chains-and-mixing-times-2/ https://pages.uoregon.edu/dlevin/MARKOV/mcmt2e.pdf