I wonder if there is a general approach to computing the mean value $\mu(t)$ of a Markov process without going through the computation of the full probability distribution.
I remember (vaguely) that Langevin did this for the mean square displacement (MSD) in a continuous random walk. He proved that the MSD grows linearly in time by deriving a differential equation for it. Perhaps there is also a method to obtain differential equations for $\mu(t)$ in discrete Markov processes?
I'm considering a process with $M$ states and an initial condition $P(n,0) = P_0(n)$. The transition matrix $T$ is nontrivial and $P(n,t) = \sum_{n=1}^{M} (T^t)_{nk} P_0(k)$ is a highly complicated function. But what I need is just the mean value $\mu(t) = \sum_{n=1}^{M} \sum_{k=1}^{M} n (T^t)_{nk} P_0(k)$ (computed, moreover, for large values of $M$).
I'm thinking that if we could derive a differential equation for $\mu(t)$, or for some transform of this function, we would bypass solving for the probability distribution. I feel that this approach must have been used at least for some special Markov chains, and if I knew where to look I might draw inspiration from there.
(I hope the question isn't too vague or naive, please tell me if it is!)