Existence of an invariant distribution and explosions in continuous time Markov chains

496 Views Asked by At

Consider an irreducible continuous time Markov chain defined on a countable state space with an infinitesimal generator Q.

Assume the generator has an invariant distribution $\pi Q=\pi$, and assume for every i, $\pi_i>0$ and $\sum_i \pi_i =1$.

Can this chain be explosive?

1

There are 1 best solutions below

4
On BEST ANSWER

This summarizes my comments above: The condition given in the question is almost sufficient for a steady state to exist. You also need a regularity condition that says there are only a finite number of transitions in finite time. For the regularity condition, it is enough to have a (finite) uniform bound on the sum transition rates out of any state.

For a counter-example that does not satisfy the regularity condition: Consider an irreducible birth-death chain with state space $S = \{0, 1, 2, ...\}$ and transition rates: \begin{align} q_{i,i+1} &= 2(4^i) \quad \forall i \in \{0, 1, 2, ...\} \\ q_{i,i-1} &= 4^i \quad \forall i \in \{1, 2 , 3, ...\} \end{align} You can show there are positive probabilities $\pi_i$ that satisfy the detail equations $\pi_i q_{i,i+1} = \pi_{i+1}q_{i+1,i}$ for all $i\in S$. However, the embedded discrete time chain is a random walk that is more likely to step forward than backward, and so the embedded chain $\rightarrow \infty$ with prob 1. Intuitively, we can understand it this way: the continuous time chain traverses the entire infinite state space in finite time, and so the $\pi_i$ probabilities can (roughly) be viewed as expected fractions of time being in state $i$ over the finite simulation.

A link to a formal statement of the steady state theorem for CTMCs (including the regularity conditions) is given in the above link.