Find mean recurrence time for an infinite homogeneous Markov chain

249 Views Asked by At

Consider the following Markov chain, which has an infinite number of states $s_1, s_2, \ldots$, with transition probabilites $$P_{1,1} = \frac 1 2,\quad P_{1,2} = \frac 1 2,\quad P_{j,1} = \frac{1}{j+1}, \quad P_{j,j+1} = \frac{j}{j+1}, \quad (j\geq 2).$$

I would like to find the mean recurrence time $s_1$ and how that it is a null recurrent state.

My thoughts:

One option is to find the eigenvectors of $P^T$ and see which eigenvectors is associated with eigenvalue 1. This then gives me the vector of recurrence times. But I'm not quite sure how to calculate this? Any thoughts? I know that if I see that the mean recurrence is $\infty$, then we have a null recurrence

1

There are 1 best solutions below

0
On BEST ANSWER

The Markov chain is recurrent since $P(1,1)=1/2>0$ and for $j>1$, $$ P^n_{jj} = \frac1{j+1}\prod_{i=1}^{j-1} \frac i{i+1} = \frac 1{j(j+1)}>0. $$

If $\nu$ is an invariant measure for $P$, i.e. $\nu=\nu P$, then we have \begin{align} \nu(0) &= \sum_{j=0}^\infty \frac1{j+2}\nu(j)\\ \nu(j) &= \frac j{j+1}\nu(j-1),\quad j\geqslant1, \end{align} so by recursion $$ \nu(j) = \prod_{i=1}^j \frac i{i+1}\nu(0) = \frac 1{j+1}\nu(0). $$ This implies that $$ \sum_{j=0}^\infty \nu(j) = \sum_{j=0}^\infty \frac1{j+1}\nu(0)=+\infty, $$ and so no invariant distribution can exist. It follows that the Markov chain is null recurrent, so the mean recurrence time to any state is infinite.