Detail of a proof of the ergodic theorem for Markov chains.

554 Views Asked by At

I want to understand the proof of this ergodic theorem written here (page 3):

Let $\{X_n\}_{n=1}^{\infty}$ be a positive recurrent, irreducible Markov chain, with state space $I$. Let $f:I\rightarrow\mathbb{R}$ be a bounded function. Then $$\lim_n\frac{1}{n}\sum_{k=0}^{n-1}f(X_k)=\sum_{i\in I}f(i)\pi_i\;\;\text{ a.s.},$$ where $\pi=(\pi_i)_{i\in I}$ is the invariant distribution.

I copy part of the proof (with more details):

Dividing both sides of the equality by a bound for $f$, we may assume that $|f|\leq1$. Let $J$ be a finite subset of $I$. First of all, notice that $$\frac{1}{n}\sum_{k=0}^{n-1}f(X_k)=\frac{1}{n}\sum_{k=0}^{n-1}\sum_{i\in I}f(i)1_{\{X_k=i\}}=\frac{1}{n}\sum_{i\in I}f(i)\sum_{k=0}^{n-1}1_{\{X_k=i\}}=\frac{1}{n}\sum_{i\in I}f(i)V_i(n),$$ where $V_i(n)$ is the number of visits to state $i$ before time $n$. Then \begin{align*}\left|\frac{1}{n}\sum_{k=0}^{n-1}f(X_k)-\sum_{i\in I}\pi_if(i)\right|= {} & \left|\sum_{i\in I}\left(\frac{V_i(n)}{n}-\pi_i\right)f(i)\right|\\ \leq{} & \sum_{i\in J}\left|\frac{V_i(n)}{n}-\pi_i\right|+\sum_{i\notin J}\left(\frac{V_i(n)}{n}+\pi_i\right) \\ \stackrel{(*)}{\leq} {} & 2\sum_{i\in J}\left|\frac{V_i(n)}{n}-\pi_i\right|+2\sum_{i\notin J}\pi_i.\end{align*}

Where does inequality $(*)$ come from?