A question about irreducible, reversible Markov chains and their time-continuous counterpart

33 Views Asked by At

So let $(X_n^x)_{n\in \mathbb{N}}$ be a $\pi$-reversible, irreducible Markov chain on some state space $E$ with transition probabilities $q(x,y)$ with starting point x. Note that $\pi$ is the stationary distribution.

Define the operators (for $f:E \to \mathbb{R}$) $$ [Qf](x) = \sum_{y \in E}f(y)q(x,y) \\ [Lf](x) = [(Q - I)f](x) = \sum_{y \in E}(f(y) - f(x))q(x,y) $$

These operators are self-adjoint and contractions on $L_2(\pi)$. Now the paper I'm reading says that we can implicitly define the time-continuous Markov chain $(Y_t)_{t \ge0}$ by the "Fokker-Planck equation" $$\frac{\partial}{\partial t}[P_{s,t}f](x) = [P_{s,t}[Lf]](x) \;\;\;\;\;\;\ s>t$$ $$[P_{s,s}f](x) := f(x)$$

with $[P_{s,t}f](x) = \sum_{y \in E}f(y)P(x,y)$ being the operator defined by $P_{s,t}(x,y)$ which is the transition probability for starting at time $s$ and point $x$ and arrive at time $t$ at $y$. So these transition probabilites by this differential equation. The paper says the following on how to interpret the new time-continuous Markov chain: "If the process is at site x, then it waits an exponential time with mean one and at the end of that time it chooses a site y with probability $q(x,y)$ and attempts to move there."

The paper I'm refering to. My question now is how do they come up with this interpretation of the new Markov chain by looking at this differential equation from above? I think I also miss some knowledge on this functional analysis approach to study Markov chains and I'd appreciate some literature tips, if you have some.