How would you find stationary distribution for the Markov chain defined by astochastic matrix?

Question

I'm not really sure how to find the stationary distribution for the matrix $P_1$. I've tried Gaussian elimination but that doesn't seem to work. I've also tried converting it into a system of equations, but I can't seem to calculate it properly. Thanks for the help :) \begin{align} P_1=\begin{bmatrix} 0 & 0 & 0 & \frac{1}{4} & 0 & \frac{1}{2} & \frac{1}{3}\\ 0 & 0 & 0 & \frac{1}{4} & 0 & 0 & 0 \\ 0 & \frac{1}{2} & 0 & 0 & 1 & 0 & 0 \\ 1 & \frac{1}{2} & 0 & 0 & 0 & \frac{1}{2} & \frac{1}{3}\\ 0 & 0 & 1 & 0 & 0 & 0 & \frac{1}{3}\\ 0 & 0 & 0 & \frac{1}{4} & 0 & 0 & 0\\ 0 & 0 & 0 & \frac{1}{4} & 0 & 0 & 0 \end{bmatrix}. \end{align}

Answer 1

The matrix you have provided is column stochastic, meaning each column sums to 1. This means that if this matrix is the transition matrix of a Markov chain, then the probability of going from state i to state j is $P_1[j][i]$. Consequently, the probability vector over the states of the Markov chain π is going to be a column vector and we will have $\pi_{t} = P_1\pi_{t-1}$.

Note that this is a bit unconventional. The transition matrix is usually taken to be row stochastic and the probability vector over the states is a row vector such that $\pi_t = \pi_{t-1}P_1$. This doesn't change the problem setting in any meaningful way, so I will adopt your convention so the explanation is easier to follow.

The stationary distribution of the Markov chain is the long-term probability distribution over states, which holds for any choice of $π_0$. Therefore we are looking for $\lim\limits_{t \to \infty}π_t= P_1\lim\limits_{t \to \infty}{π_{t-1}}$, that is, we are looking for a vector π that remains unchanged after application of $P_1$. This is the same as looking for the eigenvector $π_{\infty}$ of $P_1$ with eigenvalue 1: $P_1π_{\infty} = 1 \cdot π_{\infty}$, with the additional constraint that the elements of $π_{\infty}$ sum to 1.

Thankfully, eigenvectors are unique up to scaling, so we can just compute the eigenvector as usual and normalise it to be a valid probability distribution. Straightforward computation should yield the eigenvector $π_{\infty} = \begin{bmatrix} 0 & 0 & 1 & 0 & 1 & 0 & 0 \end{bmatrix}^T$.

We then normalise to get the stationary distribution $π_{\infty} = \begin{bmatrix} 0 & 0 & 0.5 & 0 & 0.5 & 0 & 0 \end{bmatrix}^T$

This should be immediately obvious if you draw the Markov chain, as states 3 and 5 form an absorbing cycle and they are also reachable from any other state meaning that eventually any walk on the chain will reach one of these nodes and will not be able to escape the cycle, thus forcing the distribution to be uniform over the involved states.