How can a Markov chain have more than one but a finite amount of stationary distributions?

Question

Here's my understanding of it: Assume we have an $n\times n$ stochastic matrix $P$ that represents our Markov chain such that $x$ and $y$ are stationary distributions for $P$. Then

$P(x) = x$

$P(y) = y$

$P(ax+by) = P(ax) + P(by) = aP(x) + bP(y)$ where $ax+by$ is a convex linear combination

$ = ax + by$

meaning that $ax+by$ is a stationary distribution, so there is an infinite amount of stationary distributions of P if there are at least 2.

Does this mean a Markov chain either has one or infinitely many stationary distributions?

Answer 1

Yes. Let $\mu$ and $\nu$ be two distinct stationary distributions. Now choose randomly between $\mu$ and $\nu$ with probabilities $p$ and $1-p$, and whatever distribution is chosen, choose according to it an initial state. Then the distribution of the state at time $0$ is $p\mu+(1-p)\nu$. If we chose $\mu$, then our distribution at time $1$ is still $\mu$ (because $\mu$ is stationary), and if we chose $\nu$ our distribution at time $1$ is $\nu$. Therefore our distribution at time $1$ is still $p\mu+(1-p)\nu$, so $p\mu+(1-p)\nu$ is stationary. This is just a probabilistic proof of what you proved algebraically.

Answer 2

Yes, the set of stationary distributions will always be convex, for the reason you give.

Some Markov chains (like simple random walk on the integers) have no stationary distributions. But finite MCs have stationary distributions, so your statement about "1 or infinitely many" is correct, in this case.