Expected value of a markov history chain

Question

What is $E[X_{n+1}|X_0 = i_0, . . . , X_{n−1} = x_{n−1}, X_n = i]??$ Given that $\sum_j^n jp_{i,j}=i$.

Answer 1

Hint: Well, since it's a Markov chain, $$\ E\left[X_{n+1}\,|\,X_0 = i_0, \dots , X_{n−1} = x_{n−1}, X_n = i\right]= E\left[X_{n+1}\,\left|\, X_n = i\right.\right]\ .$$ That is, you can ignore all the conditioning equations except the last. And $$ E\left[X_{n+1}\,|\, X_n = i\right]=\sum_{j=1}^njP\left[X_{n+1}=j\,|\,X_n=i\right] $$ Do you know how to express $\ P\left[X_{n+1}=j\,|\,X_n=i\right]\ $ in terms of the transition matrix (whose entries you've implicitly given as $\ p_{i,j}\ $)?