Unique stationary distribution (or measure?) of a Markov Chain

1.1k Views Asked by At

Let $(X_n)_{n \geq 0}$ be a irreducible, positive recurrent Markov chain. We have a theorem that states that the unique stationary distribution is then given by $$\pi(x)= \frac{1}{E_x[H_x]},$$ where $E_x[H_x]$ is the expected hitting time of $x$, when started at $x$, where the hitting time is given by $$H_x:=\inf\{n \geq 1 \colon X_n = x\}.$$

Is this clear that this forms a distribution? Is it clear that it forms a measure at all? How can one see this?

Context:

We are proving for a homogeneous irreducible Markov Chain on a countable state space $E$ that the following are equivalent.

1) some state is positive recurrent.

2) all states are positive recurrent.

3) there exists a stationary distribution, given by $\pi(x)= \frac{1}{E_x[H_x]}$

To prove 1) implies 2), we show that $\nu(x):=\frac{1}{E_x[H_x]}$ satisfies $\nu(x)=\sum_{y \in E} \nu(y) r(y,x)$, so that it is stationary measure. From this it follows by induction and by contradiction that all states must be positive recurrent, as the $r$ are positive (using that positive recurrent iff $E_x[H_x]= \infty$, given that it is a recurrent class).

To prove 2) implies 3), we can take the main part from 1) implies 2), and only need to show that the stationary measure can be transformed in a distribution, so we show that it is non-negative, has total mass $<\infty$, such that we can divide through the total mass.

We proved that $\nu(x)$ is a stationary measure based on the average asymptotics that $$\frac{1}{n} \sum_{k=1}^n E_y[H_x] \rightarrow \frac{1}{E_x[H_x]} \text{ as } n \rightarrow \infty$$ for a recurrent state $x$ and also using Fatou's Lemma.