On the Markov Chain Theorem

27 Views Asked by At

For an ergodic Markov chain with the transition probability matrix $P$, the Markov chain theorem stats that the limiting probabilities $\pi_{j}$ (i.e. the long-run proportions) can be computed by the linear system

$$\pi_{j}=\sum_{i=0}^{\infty}\pi_{i}P_{ij}$$ and $$\sum_{i=0}^{\infty}\pi_{i}=1$$ This means that $limit_{n\rightarrow\infty}P^{n}$ exist and can be found by solving the above linear system.

If the entries of the transition probability matrix depend on $n$ (we denote this matrix by $P^{[n]}$), then if $limit_{n\rightarrow\infty}P^{[n]}=P$, how to compute the corresponding limiting probabilites? Is there a result similar to the ergodic Markov chain theorem? In other words, how to compute $limit_{n\rightarrow\infty}\prod_{k=1}^{n}P^{[k]}$?