Markov Chains: Show that for the transient state j, $E[N_j|X_0=i]=r_{ij}/(1-r_{jj})< \infty$

38 Views Asked by At

I am unsure really where to even start with this problem. The problem comes with a few hints which are as follows:

"We know that $P(N_j=\infty|X_0=i)$ is what, and $P(N_j=m|X_0=i)$ is what? Recall the geometric series $\sum_{m=0}^{\infty}x^m=1/(1-x)$, for $|x|<1$. How can you use this to find $\sum_ {m=0}^ {\infty}mx^{m-1}$?"

We know that $P(N_j=\infty|X_0=i)=r_{ij}$ and $P(N_j=m|X_0=i)=r_{ij}*r_{jj}^{m-1}*(1-r_{jj})$. How do we use these to solve the geometric series and prove that for the transient state j, $E[N_j|X_0=i]=r_{ij}/(1-r_{jj})< \infty$?