I am looking for some textbook or paper that treats this question:
Let be $X_{1}, X_{2}, \ldots$ the random variables from a Markov Chain (MC). Is there any Central Limit Theorem (CLT) envolving their empirical distribution?
Trivial case: When the stationary distribution is the initial distribution, we know that the random variables of a MC are independent and identically distributed with $F$. If $F_{n}$ is the empirical distribution, we will have the classical CLT, $$\frac{\sqrt{n}[F_{n}(x)-F(x)]}{\sqrt{F(x)[1-F(x)]}}\stackrel{D}{\longrightarrow}N(0,1).$$
CONTEXT:
Let $X_{1},\ldots, X_{n}$ be independent identically distributed random variables, defined in some $(\Omega, \mathcal{F},P)$, distributed with $F$. This collection can be eated like a sample random of $F$. The empirical distribution function of the random sample $X_{1},\ldots, X_{n}$ is defined by $$F_{n}(x,\omega) = \frac{\displaystyle{\sum_{i=1}^{n}I_{(X_{i}(\omega)\leq x)}}}{n}.$$ When the parameter $x$ is constant, the above expression represents a random variable. Indeed, we can use the symbol $F_{n}(x)$ to denote such random variable, and rewrite our definition this way: $$[F_{n}(x)](\omega) = \frac{\displaystyle{\sum_{i=1}^{n}I_{(X_{i}(\omega)\leq x)}}}{n}.$$