Finite-state Markov chain

Question

Suppose $X_n$ is a Markov chain with transition probability matrix $p$ where the set of possible states is $S = \{1,2, \ldots, k\}$. If we are given, $X_1$, $X_2, \ldots, X_{2000}$, can we say about how many of them are equal to 1?

Answer 1

Basically no, because this is sensitive to the distribution of $X_1$, which must be given initially. However, typically you could do it if you instead had $X_N,X_{N+1},\dots,X_{N+M}$, where $N$ and $M$ are both large. Then under generic (but not universal) circumstances, there is a unique invariant distribution, the chain converges to this distribution, and $X_N,X_{N+1},\dots,X_{N+M}$ will be approximately a random sample from this distribution, in some sense. This will occur provided the chain is aperiodic and irreducible. It will actually occur even if the chain is just aperiodic, but if the chain is reducible then the invariant distribution is non-unique even though the limiting distribution is well-defined. In this case the answer depends on the initial condition, and a bit of additional work is required to identify it. By the way, this is the idea behind Markov Chain Monte Carlo sampling techniques, such as the Metropolis-Hastings method.

A warning: $N$ may need to be very large indeed if the transition probability matrix has eigenvalues which are nearly 1 in magnitude.

Edit: the above was with the first 20 terms, now the question has been edited to be the first 2000 terms. Now the answer is still no in general, but for "typical" cases where the nontrivial eigenvalues are not especially close to 1 in magnitude (say, less than 0.9 in magnitude), the transient behavior (say, the first 100 terms) will be mostly averaged away, while the invariant distribution $\pi$ will be dominant thereafter. Thus the fraction of the $X_i$ with $X_i=1$ should be approximately $\pi(1)$.