My question comes from the book "Mathematical Aspects of Mixing Times in Markov Chains". Specifically, the authors stated Lemma 1.3 in the figure above but does not include a proof for this lemma. I am wondering how can this lemma be justified in a rigorous manner. I feel like it is just a Fokker-Planck equation, but I am really stuck at the rigorous proof of the fact that $h_t = H^*_t h_0$. Thanks for any help!
I end up writing my own answer. The proof is very similar to the corresponding case in the discrete time. Fix $x \in \Omega$, then for any $t \geq 0$ we have that $$ (H^*_th_0)(x) = \sum_y H^*_t(x,y)\cdot \frac{H_0(\cdot,y)}{\pi(y)} = \frac{1}{\pi(x)}\sum_y H_t(y,x)\cdot H_0(\cdot,y) = \frac{H_t(\cdot,x)}{\pi(x)} = h_t(x), $$ which leads to the desired claim that $h_t = H^*_th_0$.