I have seen in two places, different requirements for a Markov chain to converge to its stationary/invariant distribution:
- Irreducibility and aperiodicity. As mentioned here
- Irreducibility and recurrence.
Is it true that under irreducibility and recurrence then the stationary distribution is also its limiting distribution?
And how does recurrence relate to aperiodicity, is one requirement stronger than the other?
For an example, see my comment on the discrete time Markov chain (DTMC) with $S=\{1,2\}$ and $P_{12}=P_{21}=1$, which is irreducible, recurrent, and periodic and has a solution $\pi = (1/2,1/2)$ to $\pi = \pi P$.
Here is my favorite steady state theorem: For a finite or countably infinite state space $S$, define a "probability vector" as a nonnegative vector $(\pi_i)_{i\in S}$ that satisfies $\sum_{i \in S} \pi_i=1$.
Theorem: Suppose $\{X_t\}_{t=0}^{\infty}$ is a DTMC with finite or countably infinite state space $S$ and transition probability matrix $P=(P_{ij})$. Suppose the DTMC is irreducible and there is a probability vector $(\pi_i)_{i\in S}$ that satisfies $$ \pi_j = \sum_{i\in S} \pi_i P_{ij} \quad \forall j \in S \quad \mbox{(stationary equations)}$$ Then
a) We must have $\pi_i>0$ for all $i \in S$.
b) Regardless of the initial condition $X_0\in S$, we have for all $i \in S$: $$ \lim_{T\rightarrow\infty}\frac{1}{T}\sum_{t=0}^{T}1_{\{X_t=i\}} = \pi_i \quad \mbox{(with prob 1)}$$ where $1_{\{X_t=i\}}$ is an indicator that is $1$ if $X_t=i$ and $0$ else.
c) Additionally, if the DTMC is aperiodic then for any $x_0\in S$: $$ \lim_{t\rightarrow\infty} P[X_t=i|X_0=x_0] = \pi_i \quad \forall i \in S$$
Other useful facts for general (possibly periodic) DTMCs with finite or countably infinite state space $S$ and (homogeneous) transition probability matrix $P=(P_{ij})$:
If the DTMC is irreducible then there is at most one probability vector solution to the stationary equations.
If the DTMC has a finite state space then there is at least one probability vector solution to the stationary equations.