I've been assigned to teach a course going over Markov chains next semester, and I never really feel I understand a topic until I understand "it (the topic) on manifolds".
So, a (homogeneous) Markov chain occurs in discrete-time and has a discrete (finite) state space. Given a state $\vec{v}_k$ with $\sum\limits_{i=1}^n v_{k_i} = 1$, the next state is given by $\vec{v}_{k+1} = \vec{v}_kP$, where $P$ is a (right) stochastic matrix, and, if the matrix $P$ is regular, the system drifts towards the open-loop steady-state vector $\vec{\pi}_{OL}$ with $\vec{\pi}_{OL} = \vec{\pi}_{OL}P$, $\pi_{OL_i} > 0$, and $\sum\limits_{i=1}^n \pi_{OL_i} = 1$.
A (homogeneous) continuous-time Markov process occurs in continuous time and and has a discrete (finite) state space. Given a state $\vec{v}_t$, there is a random amount of time $T_t$ where the system transitions to the next state and the new state vector is given by $\vec{v}_{t+T_t} = \vec{v}_tP(t+T_t)$ where $P(t+T_t)$ is a (right) stochastic matrix satisfying $P(0) = I$ and $P(t+h) = P(t)P(h)$. Under certain regularity (and homogeneity) conditions, $P(t) = \exp(tJ)$, where $\displaystyle J = \lim\limits_{h \to 0^+} \frac{P(h) - I}{h}$ is a constant jump-rate matrix with $j_{ii} = -\lambda_i$ and $j_{ij} = \lambda_ip_{ij}$ for $i \ne j$, where $\lambda_i > 0$, $p_{ij} > 0$, and $\sum\limits_{j \ne i} p_{ij} = 1$, so each row sums to 0. In general, the system drifts towards the open-loop steady-state vector $\vec{\pi}_{OL}$ with $\vec{\pi}_{OL}J = 0$, $\pi_{OL_i} > 0$, and $\sum\limits_{i=1}^n \pi_{OL_i} = 1$.
Is it then true that a (homogeneous) continuous-time Markov process with $\mathbb{R}^n$ as a state space is a Brownian motion $B_t$ with (1) $B_t$ everywhere continuous but nowhere differentiable, (2) $B_0 = \vec{0}$, (3) $B_s - B_t\ \coprod\ B_r$ for $r < t < s$, (4) $E\{B_s - B_t\} = \vec{0}$ for $s > t$, and (5) $B_t \sim N(\vec{0}, tI)$? I'm a little fuzzy on the concept of what might be a steady-state vector for Brownian motion on $\mathbb{R}^n$; maybe one needs a matrix $A$, a symmetric matrix $\Sigma$, a solution to a geometric SDE $dX_t=AX_tdt+\Sigma X_tdB_t$, and then $\pi_{OL}$ is the probability measure given by taking the the pdf $\rho$ of $X_0$ and setting $\displaystyle \pi_{OL}(A) = \int\limits_{A} \rho\ d\lambda$, where $\lambda$ is Lebesgue measure? Additionally, I know there's something about a function satisfying $\displaystyle \frac{\partial f}{\partial t} = \Delta(f)$ (the Laplacian) that comes up around here, but I'm not really to that point yet.
Finally, is it then true that a (homogeneous) continuous-time Markov process with a closed, Riemannian manifold $(Q^n,g)$ as a state space is a Brownian motion on $(Q,*)$ ($*$ is the basepoint of $Q$)? I'm a little fuzzy on the concept of what might be a steady-state vector for Brownian motion on a manifold; maybe one has some sort of vector field $\xi: Q \to TQ$ and mapping $\sigma: Q \to L_{Self-Adjoint}(TQ,TQ)$, a geometric SDE on $Q$, $dX_t = \xi(X_t)dt + \sigma(X_t)[\xi(X_t)]dB_t$, and $\pi_{OL}$ is the probability measure given by taking the the pdf $\rho$ of $X_0$ and setting $\displaystyle \pi_{OL}(A) = \int\limits_A \rho\ d\gamma$, using the the natual Riemannian volume measure space $(Q, \mathcal{B},\gamma)$ on $(Q,g)$ ($\mathcal{B}$ is the $\sigma$-algebra of Borel sets)? Additionally, I know there's something about a function satisfying $\displaystyle \frac{\partial f}{\partial t} = \Delta(f)$ (the Laplace–Beltrami operator) that comes up around here, but I'm not really to that point yet.
Edit 1: Per https://math.unice.fr/~delarue/Vortrag_Nice.pdf, a Markov process for a Brownian motion $B_t$ on $\mathbb{R}^n$ has $P_h(B_t) = B_{t+h}$. A probability measure $\pi_{OL}$ on the measurable space $(\mathbb{R}^n, \mathcal{E})$ ($\mathcal{E}$ is the Lebesgue measurable sets) then might be an invariant measure for the Brownian motion $B_t$/Markov process $P_t$ if $\pi_{OL}(A) = \Pr\{P_h(B_t) \in A\} = \Pr\{B_t \in A\}$ for all $t,h>0$ and all $A \in \mathcal{E}$. A similar notion might hold for Brownian motions on manifolds. See http://airvigilante194.sdf.org/mathjax/Markov.html for some sort of exposition or another on how one might "flow" a generic probability distribution $\mu$ on $\mathbb{R}^n$ to an invariant probability distribution $\pi_{OL}$.
Edit 2: Per a comment by Chris Janjigian, Brownian motions on $\mathbb{R}^n$ don't have invariant probability measures, but Brownian motions on a closed manifold $Q^n$ do; it is likely that Brownian motions on the tangent bundle $M = TQ$ of $Q$ (my main line of research interest - I'm actually learning this for a class I'm going to teach, though) similarly don't exist. The method of "flowing" a generic probability distribution $\mu$ to an invariant probability distribution $\pi_{OL}$ on a closed manifold $Q$ in the above link should still be valid.