Interpreting the infinitesimal increment of a finite state continuous time markov chain - a question concerning generators

80 Views Asked by At

Consider a markov chain on the state space $\{a,b\}$

the rate at which a particle jumps from $a$ to $b$ is $1$

the rate at which a particle jumps from $b$ to $a$ is $10$

I am trying to compute (and interpret) the evolution of the transition matrix of this markov chain.

$$\frac{d}{dt}P_t = L P_t $$

context

First step, let $[P_t(i,j)]_{i,j\in \{a,b\}}$ be the transition matrix. For instance $P(t,a,b)$ represents the probability that the chain jumps from $a$ to $b$ in a time interval of $t$.

One such matrix must satisfy the following relation (this follows from the markov property - using conditional expectations) $$ P_{t +h}(i,j) = \sum_k P_h(i,k)P_t(k,j) $$

which can be read as

The probability of going from site $i$ to site $j$ in a time interval of $t + h$ is the same as the probability of going from $i $ to any intermediate site $k$ in a time interval $h$ times the probability of going from this site $k$ to the site $j$ in a time interval $t$

using this we compute

$$\lim_{h \to 0} \frac{P_{t+h} - P_t}{h} = \lim_{h \to 0}\frac{P_{h} - P_0}{h} P_t$$

So we only need to compute $\lim_{h \to 0}\frac{P_{h} - P_0}{h} $ to find out the differential equation that governs the evolution of $P_t$.

since $$P(h,a,a) = (1 - e^{-h}) + error(h)\\ P(h,b,b) = (1 - e^{-10 h}) + error(h)$$

where $error(h)$ is such that $\frac{error(h)}{h} \xrightarrow[h \to 0]{} 0$

we can compute

$$L = \lim_{h \to 0}\frac{P_{h} - P_0}{h} = \begin{bmatrix} -1 & 1 \\ 10 & -10 \end{bmatrix}$$

Now, when we compute the infinitesimal increment of $P_t$ we get

$$ \tag{*} \frac{d}{dt}P_t(a,a) = - P_t(a,a) + P_t(b,a) $$ $$ \tag{**}\frac{d}{dt}P_t(b,a) = 10 P_t(a,a) -10 P_t(b,a) $$

The problem here is to interpret, it would make more sense that

the increment of $P_t(b,a)$ would be given by a combination of the rate at which the particle leaves the site $b$ minus the rate at which it leaves the site a,

that is, one would intuitively expect that

$$ \frac{d}{dt}P_t(b,a) = 10 P_t(b,b) - P_t(b,a)$$

But this is not the case.

How can we interpret $(*)$ and $(**)$ according to some rationale using rates of transition?

remark

We can compute explicitly $P_t$ in this case: indeed

Since $P_0(i,j) = \delta_{i,j}$, we conclude that

$$P_t = e^{t L} = \frac{1}{11} \begin{bmatrix} 10 + e^{-11 t} & 1 - e^{-11 t} \\ 10 - 10 e^{-11 t}& 1 + 10 e^{-11 t} \end{bmatrix} $$

1

There are 1 best solutions below

0
On BEST ANSWER

To properly interpret the term $\frac{d}{dt} p(t,b,a)$ you need to notice that

$$ \frac{d}{dt} e^{tL} =\frac{d}{dt} P_{t} = L P_t = P_t L $$

Use the relation $\frac{d}{dt} P_t = P_t L$, you will find that

$$\frac{d}{dt} P_t = \begin{bmatrix}p(t,a,a) &p(t,a,b)\\ p(t,b,a) &p(t,b,b) \end{bmatrix} \begin{bmatrix}-1 & 1\\ 10 & -10 \end{bmatrix}$$

$$ \frac{d}{dt} p(t,b,a) = 10 p(t,b,b) - p(t,b,a) $$

confirming your intuition.

If you have an initial measure $(0,1)$ meaning that all the mass is at $b$ then

$$\begin{bmatrix}0 & 1 \end{bmatrix}\begin{bmatrix}p(t,a,a) &p(t,a,b)\\ p(t,b,a) &p(t,b,b) \end{bmatrix} = \begin{bmatrix} p(t,b,a) &p(t,b,b) \end{bmatrix}$$

So to obtain the mass vector at time $t$ $\mu(t)$ we need to pre multiply our initial measure $\mu(0)$ by $P_t$ $$\mu(t) = \mu(0)^*P_t$$ if we want to know about its infinitesimal increment then we need to consider

$$ \lim_{h\to 0}\frac{\mu(t + h) - \mu(t)}{h} = \lim_{h\to 0}\frac{\mu(t)P_h - \mu(t)}{h} = P_t L$$