$\sum E_{x} |V_k - V_{k-1}|I\{\tau_C \ge k\} = \sum E_x[|E[(V_k - V_{k-1})|\mathscr{F}_{k-1}]I\{\tau_C \ge k\}|]$ for a submartingale Markov chain.

33 Views Asked by At

Let $V$ be a non-negative function that satisfies $$E_x V(y) - V(x) \ge 0, \; x\in C^c.$$ Let $\tau_C := \min\{n\ge 1: \Phi_n \in C\}$, where $\Phi_n$ is a Markov chain. Let $V_k = V(\Phi_k)$. In this case, how do we get the following equality? $$\sum_{k=1}^\infty E_{x} |V_k - V_{k-1}|\mathbb{I}\{\tau_C \ge k\} = \sum_{k=1}^\infty E_x[|E[(V_k - V_{k-1})|\mathscr{F}_{k-1}]\mathbb{I}\{\tau_C \ge k\}|].$$

I can see that on $\{\tau_C \ge k\}$, $E[V_k - V_{k-1}|\mathscr{F}_{k-1}] \ge 0$ and $\{\tau_C \ge k\}$ is $\mathscr{F}_{k-1}$- measurable, so we can take it out, but then the RHS is equal to $\sum_{k=1}^\infty E_x[E[(V_k - V_{k-1})|\mathscr{F}_{k-1}]\mathbb{I}\{\tau_C \ge k\}]$. I can't see how this is equal to $\sum_{k=1}^\infty E_{x} |V_k - V_{k-1}|\mathbb{I}\{\tau_C \ge k\}$. I would greatly appreciate any help.