Discrete Markov Transition Matrix

342 Views Asked by At

Well, I've been reading over the internet but I've been unable to find a straight answer. I've got a transition matrix for a Markov Discrete Chain. I've made the graph and according to my knowledge, this graph should be periodic. If the graph is periodic, then I shouldn't be able to find a stationary distribution by elevating the matrix times infinity right? Well, I added the matrix to MatLab, elevated it time 100, 1000,10000 and everytime I'm getting the same answer, even though I should be getting different ones. Can someone help me?

\begin{bmatrix} 0&0.4 &0 &0 &0 &0.6 \\ 0.3&0 &0.7 &0 &0 &0 \\ 0&0.5 &0 &0 &0.5 &0 \\ 0& 0 & 0.1 & 0 &0.9 & 0\\ 0.6&0.2 &0 &0 &0 &0.2 \\ 0&0 &0 &0.2 &0.8 &0 \end{bmatrix}

2

There are 2 best solutions below

2
On BEST ANSWER

You can return to state $1$ in two steps ($1\to2\to1$) or in three steps ($1\to6\to5\to1$) with positive probability. The period of the chain divides both 2 and 3, hence the period is 1. That is, the chain is aperiodic.

1
On

Compute the eigen-decomposition of your matrix. If you get one eigenvector that has eigenvalue $1$ and all other eigenvectors have eigenvalues of magnitude less than one, then your matrix is not periodic and any starting condition will converge to a single stationary point for the matrix.