j is null-recurrent IFF$\sum_{n=1}^{\infty}p_{jj}^{n}=\infty$ and $p_{jj}^{n}\rightarrow 0$ as $n\rightarrow \infty$

81 Views Asked by At

Let a Markov Chain be irreducible and aperiodic.

j is null-recurrent IFF $\sum_{n=1}^{\infty}p_{jj}^{n}=\infty$ and $p_{jj}^{n}\rightarrow 0$ as $n\rightarrow \infty$.

Suppose $\sum_{n=1}^{\infty}p_{jj}^{n}=\infty$.

Since $\sum_{n=1}^{\infty}p_{jj}^{n}$ diverges and using the fact that $\sum_{n=1}^{\infty} z^{n}$ diverges for $\left | z \right | \geq 1$, we have that $p_{jj}^{n}=1$. We rule out $p_{jj}^{n}>1$ the probability cannot be greater than 1.

$p_{jj}^{n}=1$ is the probability transition a Markov Chain at an initial state j visits state j in step n with probability 1. The probability transition that a Markov Chain at an initial state j does not visits state j in step n is $1-p_{jj}^{n}=0$. Indeed, j is a recurrent state as a Markov Chain at j would return to j with probability 1.

To show that j is null-recurrent requires one to show that the mean recurrence time $\mu_{jj}=E\left [ T_{jj} \right ]=\sum_{n=1}^{\infty}nf_{jj}^{n}=\infty$

Recall:

$p_{ij}^{n}=\sum_{m=1}^{n}f_{ij}^{m}p_{jj}^{n-m} \forall m\in\left [ 1,n \right ]$ where $f_{ij}^{m}$ is the probability that a Markov Chain at an initial state i visits state j for the first time in n-steps.

Since i=j and using $p_{jj}^{n}=1$: we get $1=p_{jj}^{n}=\sum_{m=1}^{n}f_{jj}^{m}p_{jj}^{n-m}$

At this point I am stuck but I think Abel theorem for countable space may pave the way forward. Any hints are appreciated.

Edit:

$p_{jj}^{n}=1$ so $\left(p_{jj}^{n}\right)^{-m}=1^{-m}=1$