I must prove that if $ X_{n} $ is a Markov chain and $ S $ is countable (at most) then $ h(X_{n}) $ is martingale, where $ h $ is harmonic, that is $ h(x) = \sum_{y \in S} P_{xy} h(y) $ (my chain is discrete time) and it is also known that $ h (X_{n}) $ is $ \mathcal{F}_n-$ adapted and integrable. Any ideas are welcome.
Context:
I have looked for some solutions on this site but it is difficult for me to understand them because of the novelty of the subject. Many argue that:
$$ \mathbb {E} (h (X_{n + 1}) | \mathcal{F}_{n}) = \sum_{y \in S} P(X_{n}, y) h (y) = h (X_{n}) $$ and this is supposedly true but I don't see why these equalities are true or what happened with $\mathcal{F}_{n} $. Thanks
Recall that Markov property states that $$\mathbb{E}(h(X_n)|F_s) = E(h(X_n)| \sigma(X_s))$$ for all $f$ that is bounded and measurable (and $F_s$ stands for the filtration at time $s$). This gives you the first equality. The second equality is by definition of the function $h$.