Transition Matrix and Invariant Probability

Question

Given the transition matrix for a 2 state Markov Chain, how do I find the n-step transition matrix P^n? I also need to take n--> inf and find the invariant probability pi?

Answer 1

A common way to find $P^n$ is to diagonalize your matrix. Then you will have $P=MDM^{-1}$ with D a diagonal matrix, so $P^n=MD^nM^{-1}$. So taking $n \rightarrow \infty$ will be easy.

Also, if $\mu$ is a measure of probability on your two states MC, $\underset{n\to \infty}\lim\mu P^n$, if it converges, is an invariant probability.

Answer 2

The conditions that have to be fulfilled are for an stationary distribution on a finite markov chain to exist are:

• It is irreducible
• Additionally if it is aperiodic then $P^n$ will converge against a projection matrix $ e \cdot \pi^T $ where $e = (1,\dots,1) $ and $ \pi $ is the stationary distribution (which is the same as primitive (for proof see Seneta 1981))

Then as nico already said - if it is aperiodic and irreducible - you can take the Jordanian Normalform and let $ n \rightarrow \infty$.