I am newbee in the field of stochastic process and I am reading a research paper wherein the following expression is given which I am not getting clearly.
Consider a discrete time Markov chain having state space $S$. Let the transition probability from state $m$ to state $n$ be $p_{mn}$.
- Given a function $V: S\rightarrow \mathfrak{R}$, the drift in state $m$ is defined as follows
$ \triangle V(m) = \sum_{n\in S}p_{mn}(V(n)-V(m))$ -----(1)
Note that, I had understood other part, except the one which starts at point 1 and goes upto eq (1).
Any help in this regard will be highly appreciated.
The idea seems to be that given a function $V$ of interest over your states, you want to define a measure for the expected amount you will drift in terms of $V$ when taking one step in the Markov chain starting from state $m$.
Indeed, starting from state $m$ and going to some other state $n$, the drift in terms of $V$ will (in your case) be $V(n)-V(m)$. Then, summing this drift over all possible states $n$ but weighting it according to the probability of going from $m$ to $n$ (that is $p_{mn}$), you essentially compute an expectation and end up with $\sum_{n \in S}p_{mn}(V(n)-V(m))$ as desired.