Markov chain: The probability of visiting from one to another status which both communicate and are recurrent in finite steps

24 Views Asked by At

Hope you are well.

I get stuck in the theorem which is mentioned as Theorem 7.2. in Page. 72 of A First Course in Stochastic Processes second edition. Here is its description:

Under a Markov chain $\left(X_{t}\right)_{t=0}^{\infty}$, two status i and j communicate and are recurrent. Then,

$f^{*}_{ij} = \sum_{n=1}^{\infty} f^{n}_{ij} = 1$,

where $f^{n}_{ij} = Pr\left(X_{n}=j, X_{t}\ne j\ \forall t=1,2,\dots,n-1|X_{0}=i\right)$.

Thank you for sharing proof in advance.