Im working on the following problem:
$(X_{n})_{n\ge0}$ is a markov chain with state space $S$ and transition matrix $P$, now for a harmonic function $h(i)=\sum_{j}p_{ij}h(j)$ where $p_{ij}$ are the entries of the transition matrix, i want to calculate:
$\Bbb E(h(X_{n+1})|\mathcal A_{n})$ where the sigma algebra $\mathcal A_{n}$ is generated by $X_{0},...X_{n}$.
I just have no idea how to approach this problem, so I dont have any mentionable results to show. I would really appreciate it if somebody could post a solution to the problem.