I'm in a measure-theoretic probability class and I am studying for an upcoming exam. Here is a problem from a book that I am studying from for this exam. But I am really not sure how to solve this problem since I am new to weak measures and representations.
Consider a Markov transition function $P$ on a compact space $X$. Prove that the corresponding Markov chain has at least one stationary measure.
I am given the following hint:
(Hint: Take an arbitrary initial measure $\mu$ and define $\mu_n = (P^{*})^{n}\mu, n \geq 0$. Prove that the sequence of measures defined by $\eta_n = (\mu_0 + \cdots + \mu_{n-1})/n$ is weakly compact and the limit of a subsequence is a stationary measure.)
I am familiar with many theorems, like Prokhorov Theorem, Riesz Representation Theorem, and more. But I'm really not sure how to solve this problem. I will really appreciate any help.
I found these notes (whose source is completely unrelated to the source of the problem) online, and they have been helpful to me: https://www.math.wisc.edu/~roch/grad-prob/gradprob-notes22.pdf
But I still cannot solve the problem.
Let $\mathcal P(X)$ denote the space whose elements are Borel probability measures on $X$, with the topology of weak convergence of measures. A special case of Prokhorov's Theorem implies that when $X$ is compact, this space $\mathcal P(X)$ is also compact. In particular, all infinite sequences in $\mathcal P(X)$ possess a limit point.
You have an infinite sequence in $\mathcal P(X)$, thus existence of a limit point means that there is a probability measure $\mu\in\mathcal P(X)$ such that some subsequence of the sequence you have written converges weakly to $\mu$.
Now fix any open set $A\subseteq X$ and consider $\mu(A)-P^*\mu(A)$. Let $\{n_k\}$ denote the convergent subsequence. Observe how $$ |\mu(A)-P^*\mu(A)|=\lim_{k\to\infty}\frac{|\mu_0(A)-\mu_{n_k}(A)|}{n_k}=0, $$ since the difference of measures telescopes and $|\mu_0(A)-\mu_{n_k}(A)|\leq 1$.