Probability of a trajectory in Markov processes

Question

I need help with a simple formula! (My question is taken from here, pag 26 eq 1.112. )

Consider a Markov Process with associated Master Equation: \begin{equation*} \frac{dP\left(C,t\right)}{dt}=\sum_{C'\neq C}W\left(C,C'\right)P\left(C',t\right)-\sum_{C'\neq C}W\left(C',C\right)P\left(C,t\right) \end{equation*} where

• $C$: configuration of the system
• $P\left(C,t\right)$: probability of being in configuration $C$ at time $t$
• $W\left(C',C\right)$: transition rates

My question is: how can I prove the following formula?

\begin{equation*} P\left(C_{1},...,C_{k};t_{1},...,t_{k}\right)=Ke^{t_{k}\mathcal{W}\left(C_{k},C_{k}\right)}...e^{t_{1}\mathcal{W}\left(C_{1},C_{1}\right)}W\left(C_{k},C_{k-1}\right)...W\left(C_{2},C_{1}\right)P_{\text{eq}}\left(C_{1}\right) \end{equation*}

where

• $K$: is a normalisation constant
• $\mathcal{W}\left(C,C\right)=-\sum_{C'\neq C}W\left(C',C\right)$
• $\lim_{t\rightarrow\infty}P\left(C,t\right)=P_{\text{eq}}\left(C_1\right)$

Thanks in advance!

Answer 1

Equation (1.110) does not consider $\mathrm e^{t\mathcal W(C,C')}$ but the quantities $[\mathrm e^{t\mathcal W}](C,C')$. The difference is that the matrix $A_t=\mathrm e^{t\mathcal W}$ is defined as the sum of the matrix series $$ A_t=\sum_{n\geqslant0}\frac{t^n}{n!}\,\mathcal W^n, $$ hence, for every $(C,C')$, $$ A_t(C,C')=\sum_{n\geqslant0}\frac{t^n}{n!}\,\mathcal W^n(C,C'). $$ A little earlier in the text, $A_t$ is introduced as $$ A_t(C,C')=P(C',t\mid C,0). $$

My question is: how can I prove the following formula?

\begin{equation*} P\left(C_{1},...,C_{k};t_{1},...,t_{k}\right)=Ke^{t_{k}\mathcal{W}\left(C_{k},C_{k}\right)}...e^{t_{1}\mathcal{W}\left(C_{1},C_{1}\right)}W\left(C_{k},C_{k-1}\right)...W\left(C_{2},C_{1}\right)P_{\text{eq}}\left(C_{1}\right) \end{equation*}

This (equation (1.112)) is considering the density of a joint distribution. To understand this point, consider $\Delta_k=(\mathbb R_+)^k$ and $\mathcal C$ the set of configurations. Then the RHS of (1.112) is the joint density on $\mathcal C^k\times\Delta_k$ of the $k$ first different configurations and of the times spent in them by the system. This is explained in the paragraph of text just before (1.112).

Thus, (1.112) is correct, the exponentials in (1.112) are really $\mathrm e^{t\mathcal W(C_i,C_i)}$, not $[\mathrm e^{t\mathcal W}](C_i,C_i)$, the quantities $\mathcal W(C_i,C_i)$ in the exponentials are defined by (1.108) and each $-\mathcal W(C_i,C_i)$ represents the total rate at which the system leaves state $C_i$. According to me, the factor $K$ should read $$ K=-\mathcal W(C_k,C_k). $$ With this convention and using the notation $\xi_{i:j}$ as a shorthand for $(\xi_i,\xi_{i+1},\ldots,\xi_j)$ for every symbol $\xi$ and every integers $i\leqslant j$, note that $$ \int_0^\infty\sum_{C_k}P(C_{1:k};t_{1:k})\mathrm dt_k=P(C_{1:k-1};t_{1:k-1}). $$ In the end, and interpreting with care the various quantities involved, I see only (1.107) as problematic. Replacing (1.107) by the condition that $\mathcal W(C,C')=W(C,C')$ for every $C\ne C'$ saves the day. Of course, one can complain that the fact that $P(C_{1:k};t_{1:k})$ is a density is not even mentioned and that the notations are rather confusing, but...