Countable state space Markov chain -- positive recurrence & return time

699 Views Asked by At

Let $X_n$ be an irreducible Markov chain on a countable state space. I know that if the chain is positive recurrent, then there is a distribution $\pi$ that satisfies $$ \pi(x) = \lim_{n\to\infty} p_n(y,x), $$ independently of $y$. I also that the expected return time to a state $x$ is given by $1/\pi(x)$. This begs two questions:

  1. Is it guaranteed that $\pi(x) > 0$? As in, will it ever be the case for an irreducible (but not necessarily positive recurrent) chain to have $\lim_{n\to\infty} p_n(y,x) > 0$ for some $x$ but not others?

  2. Suppose I did not know the chain was positive recurrent, but showed that the return time was finite. Would this imply the return time for other states is also finite? Basically, does return time being finite for a single state imply positive recurrence?

If this needs to be split into two questions, I can do that.

1

There are 1 best solutions below

2
On BEST ANSWER
  1. Without positive recurrence, no, $\pi$ may not be positive. For example, if there is a transient class but nevertheless $\pi$ exists, then $\pi$ is just zero on that transient class.
  2. Positive recurrence is a class property, so if you have an irreducible chain with one positive recurrent state then all states are positive recurrent.