Markov chain generator vs. transition matrix

684 Views Asked by At

I have seen (in e.g. this paper) discussion of "Markov generators" $Q$ which are distinct from the transition matrices $P$ which I am more familiar with. One crucial difference is that $Q$ may have negative values along the main diagonal so as to enforce row-sums being $0$, this being in lieu of stochasticity. However, reversibility w.r.t. a distribution $\pi$ in that setting is put as $\pi(x)Q(x,y)=\pi(y)Q(y,x)$, where $Q$ takes the place where I usually see $P$. What is the interpretation of these objects, and how do they relate to the transition matrix formulation?

1

There are 1 best solutions below

0
On BEST ANSWER

A "Transition Matrix" is for a (time-)discrete process. It tells you the probability of going from one state to another in "one step".

A "Generator" $Q$ is for (time-)continuous process. For a time $t$ (which is a positive real number), the transition matrix is given by $\exp(tQ)$. (See here if you are unclear what the "exponential of a matrix" is supposed to be).