Is it true that a irreducible, recurrent markov chain with nonzero stationary distribution tends to stay in the same state?

236 Views Asked by At

Define $N_j = \min\{ n>0 \ : \ X_n = j\}$ and $m_j = E(N_j | X_0 = j)$. According to Sheldon Ross: Introduction to probability models page 204 in the 11th edition, if a markov chain is irreducible and recurrent, then it has $$ \pi_j = \frac{1}{m_j}. $$ That must mean if $\pi_i \neq 0$, then $$ 0<\pi_i = \frac{1}{m_i} \qquad \text{if and only if} \qquad 1> m_i\pi_i = \pi_iE(N_j | X_0 = j). $$ Can this be right? It seems wildly strange that a markov chain which as a non-zero stationary probabilities only tends to transition into the same state at each time step $n$. If it is false, can you explain what's wrong. And if it is correct, how can we see that it is not so crazy after all?

1

There are 1 best solutions below

0
On BEST ANSWER

No, that's not correct, because $N_j \ge 1$ by definition, so by monotonicity, $m_j = E[N_j \mid X_0 = j] \ge 1$.