Wald's Lemma for Markov Chains

317 Views Asked by At

Suppose $(X_n)_{n\geq 1}$ is a sequence of iid random variables, and $N$ is a stopping time defined with respect to the filtration $\mathcal{F}_{n}=\sigma(X_{1},\ldots,X_{n}), ~n\geq 1$. Then, the first theorem of Wald's states that under the additional assumption that $N$ is finite almost surely, \begin{equation} E\left[\sum\limits_{k=1}^{N}X_{k}\right]=E[N]\cdot E[X_{1}]. \end{equation}

Is there any analogue of the same equation for the case when $(X_n)_{n\geq 1}$ is a Markov process? I could find some content pertaining to this in this paper.

It would be really helpful to me if I am pointed to some more references (if any) on this.