How to find the conditional expectation of the following random variable?

Question

Let $\{N_n; n=1,2,3,\dots,\}$ be an irreducible and aperiodic Markov chain with transition probability matrix $\mathcal{P}=\begin{pmatrix} p_{0,0} & p_{0,1}&p_{0,2}&\cdots \\p_{1,0} & p_{1,1}&p_{1,2}&\cdots \\0 & p_{2,1}&p_{2,2}&\cdots \\0 & 0&p_{3,2}&\cdots \\ \vdots & \vdots&\vdots&\ddots \\\end{pmatrix}$, where $$p_{0,k}=b_k+c_k, ~ k\ge 0,$$ $$ p_{k,k-1}=R~ a_0,~ k\ge 1 ~~~~~~~~~\mbox{and}$$ $$p_{n,k}=R~a_{k+1-n}+(1-R)~a_{k-n},~n\ge 1,~k\ge n. $$ How to find $E(N_{k+1}-N_k|N_k=n)$? Any hints to proceed further?

Answer 1

Direct computations show that, for every $n\geqslant1$, $E(N_{t+1}-N_t\mid N_t=n)=\sum\limits_{k=0}^\infty ka_k-R\sum\limits_{k=0}^\infty a_k$ (assuming the series $\sum\limits_kka_k$ converges absolutely, say). This drift does not depend on $n\geqslant1$. The formula for $E(N_{t+1}-N_t\mid N_t=0)$ is different since the transitions from state $0$ are defined differently.

Edit: Recall that, by definition of the transition matrix of the Markov chain $(N_t)$, for every measurable function $G$, $$E(G(N_{t+1})\mid N_t=n)=\sum_kp_{nk}G(k),$$ where the sum runs over every possible state $k$, in the present case every $k$ in $\{0,1,2,\ldots\}$.

Edit-edit: Conditionally on $N_t=n$ with $n\geqslant1$, $N_{t+1}-n=-1$ with probability $Ra_0$ and, for every $k\geqslant0$, $N_{t+1}-n=k$ with probability $Ra_{k+1}+(1-R)a_k$, thus $$E(N_{t+1}-N_t\mid N_t=n)=-Ra_0+\sum_kk(Ra_{k+1}+(1-R)a_k).$$ The RHS is $$-Ra_0+R\sum_kka_{k+1}+(1-R)\sum_kka_k=-Ra_0+R\sum_k(k-1)a_{k}+(1-R)\sum_kka_k.$$ Can you finish?