Here is my understanding of (Discrete) Markov Chains:
Suppose there is a Discrete Markov Chain with 3 States $A$,$B$,$C$ and there are $N$ people
1) Stationary Distribution:
- Suppose at time=$n$, there are $\frac{n_i}{n_i+n_j+n_k}$ people in $A$, $\frac{n_j}{n_i+n_j+n_k}$ people in $B$ and $\frac{n_k}{n_i+n_j+n_k}$ people in C.
- $n_i$ + $n_j$ + $n_k$ = $N$
- If at time=$n+1$, there are still $\frac{n_i}{n_i+n_j+n_k}$ people in $A$, $\frac{n_j}{n_i+n_j+n_k}$ people in $B$ and $\frac{n_k}{n_i+n_j+n_k}$ people in C - then $n_1, n_2, n_3$ is considered as the Stationary Distribution
Mathematically, we can say - for a transition matrix $P$ : $\pi$ is a Stationary Distribution if $\pi$ satisfies the following :
$$\pi = \pi P$$
Solving for $\pi$ is like solving a system of linear equatiosn.
2) Limiting Distribution:
- Suppose at time=$0$, there are $\frac{n_i}{n_i+n_j+n_k}$ people in $A$, $\frac{n_j}{n_i+n_j+n_k}$ people in $B$ and $\frac{n_k}{n_i+n_j+n_k}$ people in C.
- $n_i$ + $n_j$ + $n_k$ = $N$
- After infinite time steps, there are now $m_1$ people in $A$, $m_2$ people in $B$ and $m_3$ people in C.
- $m_1$ + $m_2$ + $m_3$ = $N$
- If after infinite time steps, for all possible sets of ($n_i,n_j,n_k$) ... there are still $m_1$ people in $A$, $m_2$ people in $B$ and $m_3$ people in C : $(m_1, m_2, m_3)$ is considered as the Limiting Distribution
Mathematically, we can say - for a transition matrix $P$, any initial distribution $\pi_0$ : $\pi$ is a Limiting Distribution if the following limit exists :
$$\lim_{n \rightarrow \infty} \pi = \pi_0 P^n$$
My Question: Given a Markov Chain, it seems straightforward to obtain the Stationary Distribution ... but I am not sure how we would try and obtain the Limiting Distribution. I understanding that if you can show that a Markov Chain is Ergodic (i.e. both irreducible and aperiodic), then the Limiting Distribution is equivalent to the Stationary Distribution.
But from a general perspective, without relying on this Ergodic fact, it seems like you would need to prove that $\lim_{n \rightarrow \infty} \pi_0 P^n$ is the same for all $\pi_0$.
As such, I am not sure how this can be proven as it would require taking the limit of a matrix.
For a specific Markov Chain, I could take different values of $\pi_0$, and manually exponentiate $P$ to some large $n$ (e.g. $n$ = 100) and check if $\pi$ remains the same. But of course this is not a mathematical proof.
Thus, for a given matrix $P$, is it possible to prove that $\lim_{n \rightarrow \infty} \pi_0 P^n$ is the same for all $\pi_0$?
Thanks!
- The only understandable reference I found on this:https://www.probabilitycourse.com/chapter11/11_2_6_stationary_and_limiting_distributions.php