Just a little confusion with recurrence in Markov Chains

48 Views Asked by At

Is it possible that in a Markov Chain one can go to a null recurrent state from a positive recurrent state? Note I assume the state space to be infinite otherwise the question makes no sense. If so give an example. If not, well then how does one prove it?

1

There are 1 best solutions below

0
On

No, it is not possible.

For every state $i$ in a Markov chain, the limit of ${1\over N}\sum_{m=1}^N p_{ii}^{(m)}$ exists as $N\to\infty$. The state $i$ is called null when the limit is zero, but called positive when the limit is positive.

For two states $i,j$ and fixed $\ell,n>0$ we have $${1\over N}\sum_{m=1}^N p_{ii}^{(\ell+m+n)}\geq p_{ij}^{(\ell)}\,{1\over N}\sum_{m=1}^N p_{jj}^{(m)}\,p_{ji}^{(n)}.$$ Letting $N\to\infty$ shows that if $i$ is null then either $j$ is null, or $i$ and $j$ fail to communicate.

However, if $i$ is recurrent and $i\to j$, then $i$ and $j$ communicate. If $i$ is also null, the argument above shows that $j$ must be null as well.