Consider a Markov Chain $\{X_n : n \in \mathbb{Z}_+\}$ with general state space $(S,\S)$ (not necessarily countable) defined on the canonical product space $(\Omega, F)=(S^{\mathbb{Z}_+}, \S^{\mathbb{Z}_+} $. If f is a bounded measurable function S, let $Gf(x)= E_x(f(X))-f(x)$. G is called the generator of the Markov Chain $\{X_n : n \in \mathbb{Z}_+\}$. We say that h is a harmonic function on the set $D \in \S$ iff $Gh(x)=0$ for all $x \in D$.
a) If $h$ is a bounded harmonic function on $S$ prove that $h(X_n)$ is an $(F_n^X)$-martingale with respect to every $P_x$. If $A \in \S$ and $h$ is a bounded function on S which is harmonic on $A^c$, show that $h(X_{n \wedge V_A)}$ is an $F_n^X$-martingale with respect to every $Px$.
b) Let $A \in \S$ satisfy $P_x(V_A < \infty)=1$ for every $x \in S$. Let $f: A \rightarrow \mathbb{R} $ be bounded and measurable. Prove that $h(x)=E_x(f(X_{V_A}))$ is the unique bounded function on S which is harmonic on $A_c$ and equals f on A.
c)If $f: S \rightarrow \mathbb{R} $ is bounded and measurable, prove that $M_n^f=f(X_n)- \sum_{i=0}^{n-1}Gf(X_i)$ is an $(F)n^X)$- martingale with respect to every $P_x$.
For a) I have that $E(h(X_{n+1}|F_n)= E_{X_n}(h(X_{n-1}))$ by Markov Property. Then I'm not sure what to do to make $E(h(X_{n+1}|F_n)= h(X_n))$. And I'm not sure what to do for the rest of part a).
For b) and c), I see they involve some sort of trick using Markov Property. But I'm not sure how to proceed.
Please help. Thank you!
(a) Hint: $E(h(X_{n+1})|F_n^X)(\omega)=\sum_{y \in S} h(y)p(X_n(\omega),y)$, for transition probability $p$, is an equivalent formulation of the Markov Property.
(b) is the probabilist's interpretation of the Dirichlet Problem.
(c) Hint: $E_{X_n}(f(X_1))=f(X_{n+1})$ by the Markov Property.