This is a problem from the book "Markov chains". Let $(X_n)_{n\ge 0}$ be a Markov chain on $I$ and let $A$ be an absorbing set in $I$. Set $$T=\inf\{n\ge 0 : X_n \in A\}$$ and $$h_i = \mathbb{P}_i(X_n \in A \text{ for some } n \ge 0) = \mathbb{P}_i(T<\infty)$$ Show that $M_n = h(X_n)$ is a martingale.
This is my solution: \begin{align*} E[M_{n+1} \mid \mathcal{F}_n] = E[h(X_{n+1}) \mid \mathcal{F}_n] &= \sum_{j \in I} p_{X_n,j}h(j) \\ &= h(X_n) \text{ (First step analysis)}\\ &= M_n \end{align*} Am I correct or not? I'm afraid I missed something. Thanks!
Some notions first. To show that $h(X_n)$ is a martingale it is just sufficient to check whether $$ \mathsf E[h(X_{n+1})|X_n] = h(X_n) \iff \mathsf E_x[h(X_1)] = h(x) \iff h(x) = Ph(x) \tag{1} $$ since $(X_n)$ is a Markov process. Here $P$ is a transition operator $$ Pf(x):=\mathsf E_x[f(X_1)] = \sum_{y\in I}p_{x,y}f(y). $$ and all functions satisfying $Pf=f$ are called harmonic. As a result, $(1)$ shows that $f$ is harmonic if and only if $f(X_n)$ is a martingale.
So, your step is right where you get $$ \mathsf E[M_{n+1}|\mathscr F_n] = \sum_{y\in I}p_{X_n,y}h(y) $$ is correct. To show that the latter is $h(X_n)$, just use the formula $$ h(x) = 1_A(x)+1_{A^c}(x)Ph(x). \tag{2} $$ In general, $(2)$ describes the hitting probability for any set $A$ and then $h(X_n)$ is a supermartingale in general. It is a martingale provided $A$ is absorbing.