Markov's matrix into stationary Distribution

48 Views Asked by At

How do I know if this Markov's Transition Matrix converges into a stationary distribution?

$$P= \begin{bmatrix} .8 & .2 & 0 \\ .3 & .4 & .3 \\ .2 & .1 & .7 \\ \end{bmatrix}$$

1

There are 1 best solutions below

4
On BEST ANSWER

An irreductible aperiodic chain on a finite nomber of states $0, 1, \dots , N-1$ it's positive recurrent and is transition matrix is

$$\lim_{n \to \infty} P^{(n)}= \begin{bmatrix} \pi_0 & \pi_1 & \dots & \pi_{N-1}\\ \vdots & \vdots & \ddots & \vdots \\ \pi_0 & \pi_1 & \dots & \pi_{N-1} \\ \end{bmatrix},$$

where $(\pi_0, \pi_1, \dots, \pi_{N-1})$ is the stationnary distribution which is given by the identity $\pi_j = \mu_j^{-1}$ for $j=0, 1, \dots , N-1$

Definition : For any positive recurrent j, the quantity $\mu_j^{-1}$ is the average fraction of a long-term $j$ visits from the initial state $j$.

Otherwise, a finite or infinite countable $\pi = (\pi_j)$ is named a stationnary distribution pour an irreductible chain, therefore with one class of states, if and only if the following condition are satisfied :

  1. $\pi_j > 0$ for all $j$;
  2. $\sum_j \pi_j=1$; and
  3. $\sum_i \pi_i P_{ij}$ for all $j$, i.e. $\pi = \pi P$ in matrix notation.

The conditions $1$ and $2$ ensure that $\pi_j$ is a probability distribution strictly positive, while the condition $c$ represent the stationnarity equation.

Conclusion : In general, you could find the stationnary distribution in using $\mu_j^{-1}$ with ergodic theorem or the condition 1, 2 and 3 .