How do I know if this Markov's Transition Matrix converges into a stationary distribution?
$$P= \begin{bmatrix} .8 & .2 & 0 \\ .3 & .4 & .3 \\ .2 & .1 & .7 \\ \end{bmatrix}$$
How do I know if this Markov's Transition Matrix converges into a stationary distribution?
$$P= \begin{bmatrix} .8 & .2 & 0 \\ .3 & .4 & .3 \\ .2 & .1 & .7 \\ \end{bmatrix}$$
An irreductible aperiodic chain on a finite nomber of states $0, 1, \dots , N-1$ it's positive recurrent and is transition matrix is
$$\lim_{n \to \infty} P^{(n)}= \begin{bmatrix} \pi_0 & \pi_1 & \dots & \pi_{N-1}\\ \vdots & \vdots & \ddots & \vdots \\ \pi_0 & \pi_1 & \dots & \pi_{N-1} \\ \end{bmatrix},$$
where $(\pi_0, \pi_1, \dots, \pi_{N-1})$ is the stationnary distribution which is given by the identity $\pi_j = \mu_j^{-1}$ for $j=0, 1, \dots , N-1$
Definition : For any positive recurrent j, the quantity $\mu_j^{-1}$ is the average fraction of a long-term $j$ visits from the initial state $j$.
Otherwise, a finite or infinite countable $\pi = (\pi_j)$ is named a stationnary distribution pour an irreductible chain, therefore with one class of states, if and only if the following condition are satisfied :
The conditions $1$ and $2$ ensure that $\pi_j$ is a probability distribution strictly positive, while the condition $c$ represent the stationnarity equation.
Conclusion : In general, you could find the stationnary distribution in using $\mu_j^{-1}$ with ergodic theorem or the condition 1, 2 and 3 .