How can I show that $(f(X_n))_n$ is a martingale?

Question

Let me assume $X=(X_n)_n$ is a Markov chain with values in $E$. Assume further that $Q$ is it's transition matrix and $\nu$ the initial distribution. Let $f:E\rightarrow \Bbb{R}_+$ be a harmonic function which means that $f(x)=\sum_{e\in E} Q(x,e)f(e)$. I need tho show that if $\Bbb{E}_\nu (f(X_n))<\infty$ for each $n$ then $(f(X_n))_n$ is a martingale.

So I started as follows:

$$\begin{align} \Bbb{E}_\nu(f(X_{n+1})|F_n) &=\Bbb{E}_\nu\left(\sum_{e\in E}Q(X_{n+1},e)f(e)\big|F_n\right)\\&= \sum_{e\in E} f(y) \Bbb{E}_\nu(Q(X_{n+1},y)|F_n)\end{align} $$

Now thought about using the simple Markov property, but I don't see how. We had the following version of the simple Markov property:

Fix $n\geq 1$ consider $G:E^\Bbb{N}\rightarrow \Bbb{R}_+$ $F_\infty$-measurable and define $\Phi(x)=\Bbb{E}_x(G(X))$. Then for any probability measure $\nu$ on $E$ $$\Bbb{E}_\nu(G((X_n,X_{n+1},...)|F_n)=\Phi(X_n)$$

Could maybe someone help me further because at the end I should get that the computation above is $f(X_n)$.

Thanks for your help

Answer 1

1. Recall the Markov property

$$ P(X_{n+1} = e| {\cal F}_n) =P_{X_n} (X_1=e)=Q(X_n,e).$$

2. If $f$ is any nonnegative function, use monotone convergence to justify:

$$ E [ f(X_{n+1} | {\cal F}_n ] = \sum_{e} Q(X_n ,e) f(e).$$

3. If, in addition, $f$ is harmonic, then the righthand side is equal to $f(X_n)$.