Markov Chain with Unique Stationary Distribution

Question

Why is it that a non-negative integers Markov chain with transition prob. matrix (t.p.m) $P$ given by $p_{i,i+1} = p$ and $p_{i,0} = 1 − p$, has a unique stationary distribution $π$?

Answer 1

Let $\pi$ denote a stationary distribution for the given Markov chain, assuming for the moment such a distribution exists. We may write $\pi$ as a vector, thus:

$\pi = (\pi_0, \pi_1, \pi_2, \ldots, \pi_k, \pi_{k + 1}, \ldots ); \tag 1$

also, $\pi$ must satisfy the normalization condition

$\displaystyle \sum_0^\infty \pi_k = 1, \tag 2$

which ensures that $\pi$ is in fact a probability distribution on the non-negative integers

$\Bbb Z_\ge = \{ z \in \Bbb Z, \; z \ge 0 \}. \tag 3$

Now it will be recalled that the equations

$p_{i, i + 1} = p, \; p_{i, 0} = 1 - p \tag 4$

define the conditional transition probabilities 'twixt the specified states ($i$ and $i + 1$, $i$ and $0$ according to the indices); thus the stationary probabilities $\pi_k$ must satisfy

$\pi_{i + 1} = p \pi_i, \tag 5$

and

$\pi_0 = (1 - p) \pi_i; \tag 6$

if we apply (5) repeatedly, starting with $i = 0$ we find

$\pi_1 = p \pi_0, \tag 7$

$\pi_2 = p\pi_1 = p^2 \pi_0, \tag 8$

and it is easy to see this pattern leads to

$\pi_k = p^k \pi_0, \; k \in \Bbb Z_\ge; \tag 9$

substituting this into the normalization equation (2) yields

$\pi_0 \displaystyle \sum_0^\infty p^k = \sum_0^\infty p^k \pi_0 = \sum_0^\infty \pi_k = 1, \tag{10}$

and since

$\displaystyle \sum_0^\infty p^k = \dfrac{1}{1 - p}, \; 0 < p < 1; \tag{11}$

we may solve (10) and obtain

$\pi_0 = 1 - p, \tag{12}$

and hence from (9),

$\pi_k = p^k(1 - p). \tag{13}$

As a consistency check, we may use (13) in the equation for $\pi_0$:

$\pi_0 = \displaystyle \sum_0^\infty (1 - p)\pi_i = \sum_0^\infty (1 - p)(1 - p)p^k = (1 - p)^2 \sum_0^\infty p^k = (1 - p)^2 \dfrac{1}{1 - p} = 1 - p. \tag{14}$

Now since a distribution $\pi$ as in (1) is given by (13), we see that a stationary distribution indeed exists; furthermore, it is clear that it is uniquely determined by (9)-(13). Thus the given Markov chain defined by (4) has a unique stationary distribution.

Nota Bene: Of course we should note that the above only applies subject to the condition

$0 < p < 1; \tag{15}$

if $p = 1$, then (6) and (9) show that

$\pi_k = 0, \; k \in \Bbb Z_\ge, \tag{16}$,

which, since such $\pi$ can't satisfy (2), is inadmissible as a probability distribution; if $p = 0$, (16) again follows from (5) and (6), and again we arrive at an inadmissible $\pi$. So we need to adopt (15) to obtain a unique and sensible result. End of Note.

Answer 2

Algebraically you are trying to solve $\pi P = \pi$ (or equivalently $\pi(P-I) = 0$. Write out a couple of implied equations and you will see how to derive the solution and that it is unique...