When should I stop iteration when a transition matrix M gives the same distribution vector?

41 Views Asked by At

We know that the stationary distribution of a Markov Chain with transition matrix P is some vector, ψ, such that ψP = ψ. In other words, over the long run, no matter what the starting state was, the proportion of time the chain spends in state j is approximately ψj for all j.

However, let us say that the initial state row vector: $$ ψ_0 = [0.1667 , 0.0714, 0.1609 , 0.0252, 0.0241, 0.0512, 0.3087, 0.0149, 0.1609, 0.0149]. $$ After multiplying ψ with the matrix $M$, 3 times, we have something like this: $$ ψ_3 = [0.1705, 0.0629, 0.1696, 0.0293, 0.0282, 0.0511, 0.2806, 0.0148, 0.1696, 0.0148]. $$ Now look at the first 4 digits, when should I stop to have these number of digits remain the same when coverage?

For example, the first cell in ψ0 is different from the first cell in ψ3 when we check the accuracy of the 4 digits in both. But after, for instance, 65 times, the vectors let us say ψ58 = ψ59 = ψ60 = ... ψ65. All will be the same.

So again as we saw, I want to make somehow an analytical condition or solution to stop iterating this multiplication loop.