Markov chain - clique

Question

Is there a special name (or case) for a finite Markov chain which all states are reachable from any state with positive probability? Does anyone familiar with a problem modeled by this kind of chain?

Answer 1

To elaborate on what @Did already said, I think you are referring to the reducibility or regularity property of a Markov chain. For the definitions, see the Wikipedia article on Markov chains or chapter 11 section 3 of Grinstead and Snell's Introduction to Probability. Another closely related property is connectedness in graph theory.

Markov chains of this type are very important (specifically when they do not contain temporal cycles i.e. aperiodic) for thermodynamics and the Ergodic Hypothesis. The chain isn't irreducible unless it aperiodic in addition to all states being mutually reachable. If all states can be visited then the time average is equal to an ensemble average of microstates in equilibrium allowing for the powerful idea of statistical ensembles to become useful.

If all states are reachable (and aperiodic), it also allows for important analytic statements to be made, most importantly, the Perron-Frobenius theorem.

If you mean a Markov chain that is 1-step connected, then one can think of the Markov chain of biased dice. Another application area where 1-step connect Markov chains might be useful is in the study of random searches with search locations chosen at random with replacement.