The relationship of Markov chain properties

58 Views Asked by At

I'm trying to understand the Metropolis-Hastings algorithm and its theoretical background Markov Chain properties.

Is there something wrong with this diagram for the time-homogeneous, finite state Markov chains? (Not sure if it requires to be time-homogeneous and finite.)

Especially, I'm not sure that if a Markov chain is reversible it can be said it's ergodic or positive recurrent?

  • A circle is a property of states
  • A box is a property of Markov chains.

mc properties relationship

You might want to edit the diagram with this code using graphviz.

digraph {
    # nodes
    sAccesible [label=accessible]
    sComunicate [label=communicate]
    cIreducible [label=irreducible, shape=box]

    sAperiodic [label=aperiodic]
    sRecurrent [label="positive recurrent"]
    sErgodic [label=ergodic]
    cErgodic [label=ergodic, shape=box, fillcolor=gray, style=filled]

    sDetailedBalance [label="detailed balance"]
    cReversible [label=reversible, shape=box, fillcolor=gray, style=filled]
    cStationaryDistribution [label="has stationary distribution", shape=box]
    cUniquePositiveStationaryDistribution [label="has unique positive stationary distribution", shape=box, fillcolor=lightblue, style=filled]

    and [label="+", shape=circle]

    # edges
    cIreducible -> sComunicate -> sAccesible
    sErgodic -> sAperiodic
    sErgodic -> sRecurrent
    cErgodic -> sErgodic
    cErgodic -> cIreducible
    cReversible -> sDetailedBalance
    cReversible -> cStationaryDistribution
    cReversible -> cIreducible

    cErgodic -> and [style=dashed]
    cStationaryDistribution -> and [style=dashed]
    and -> cUniquePositiveStationaryDistribution
}