Derivation of a stationary distribution of a Markov chain

85 Views Asked by At

Consider the following markov chain in index notation

$$P_j(t+1)=\sum_{i=1}^{n}M_{ij}P_i(t)$$

where $M_{ij}$ is a column transition matrix .

I would like to develop the steady state given by $\pi M=\pi $ in an index notation for the Column transition matrix.

This is what I have so far

$$P_j(t+1)=\sum_{i=1}^n M_{ij}P_i(t)=\sum_{i=1}^n\left(M_{ij}P_i(t)+M_{ii}P_i(t)\right)=$$

$$=\sum_{i=1}^nM_{ij}P_i(t)+\sum_{i=1}^nM_{ii}P_i(t)=\sum_{i=1,i\neq j}^nM_{ij}P_i(t)+\left(1-\sum_{j=1,i\neq j}^nM_{ji}\right)P_j(t)=$$

$$=\sum_{i=1,i\neq j}^nM_{ij}P_i(t)+P_j(t)-\sum_{j=1,i\neq j}^nM_{ji}P_j(t)=$$

$$\longrightarrow$$

$$P_j(t+1)-P_j(t)=\sum_{i=1,i\neq j}^nM_{ij}P_i(t)-\sum_{j=1,i\neq j}^nM_{ji}P_j(t)$$ $$=\sum_{i=1,i\neq j}^nM_{ij}P_i(t)-\sum_{j=1,i\neq j}^nM_{ji}P_j(t)+\sum_{i=1}^nM_{ii} P_i(t)-\sum_{j=1}^nM_{jj} P_j(t)$$

$$=\sum_{i=1}^nM_{ij}P_i(t)-\sum_{j=1}^nM_{ji}P_j(t)$$

At steady state

$$\sum_{i=1}^nM_{ij}P_i(t)=\sum_{j=1}^nM_{ji}P_j(t)$$

Which is a trivial solution, I don't understand how to obtain the steady state solution $\pi=M\pi$ in matrix form the equation above.

Please advise, note that it is important for me, if possible, to remain in mattrix form for a column transition matrix.