This issue involves a very important concept, which is the $\beta$-mixing nature of stochastic processes. All the stochastic processes we discuss are time-positive and discrete. To strictly adhere to this concept, we first introduce it:
- ($\beta$-mixing) Firstly, this concept is limited to markov process $\left\{ Z_t \right\} _{t\geqslant 1}$. We assume that markov processes have a unique stationary distribution $\nu$. When $\left\{ Z_t \right\} _{t\geqslant 1}$ has exponential ergodicity, there exists $\beta$-mixing coefficients $\beta(t)$, i.e. $$ \beta \left( t \right) =\mathop {\mathrm{sup}} \limits_{k\geqslant 1}\mathbb{E} \left[ \mathop {\mathrm{sup}} \limits_{\mathcal{B} \in \mathcal{F} _{k+t:\infty}}\left| \mathbb{P} \left( \mathcal{B} |\mathcal{F} _{1:k} \right) -\mathbb{P} \left( \mathcal{B} \right) \right| \right] , $$ where $\mathcal{F} _{n:m}=\sigma \left( \left\{ Z_n,\cdots ,Z_m \right\} \right) $, and $\beta \left( t \right) \leqslant C\exp \left( -bt \right) $ w.r.t. constants $C$ and $b$. See the following reference for more detail:
Meitz, Mika, and Pentti Saikkonen. "Subgeometric ergodicity and β-mixing." Journal of Applied Probability 58.3 (2021): 594-608, Page 594.
We usually call this markov process is $\beta$-mixing.
Now we will divide the problem into two levels:
(Q1, Simple Baseline) Suppose there exists a general process $\left\{ X_t \right\} _{t\geqslant 1}$ with $\left| X_t \right|<\infty $ and $\mathbb{E} \left| X_t \right|<\infty $, $\mathbb{E} \left| X_t \right|^2<\infty $. Now truncate the process with constant time $N<\infty$. Construct a new process $\left\{ Z_t \right\} _{t\geqslant 1}$ where $Z_t=X_t$ a.s. when $t\leqslant N$, and $\left\{ Z_{N+t} \right\} _{t\geqslant 1}$ is a $\beta$-mixing markov process. Denote $\mathcal{F} _{n:m}=\sigma \left( \left\{ Z_n,\cdots ,Z_m \right\} \right) $.
Question: Suppose $\mathcal{F} _{N+1:\infty}$ is independent of $\mathcal{F} _{1:N}$, can we also claim that $\left\{ Z_t \right\} _{t\geqslant 1}$ (note it is not markovian) is $\beta$-mixing?
(Corollary of Q1) Suppose exists non-trivial constant $T\leqslant N<\infty $ satisfying $\frac{N}{T}=k\in \mathbb{N} _+$. We can "cut" $\left\{ Z_t \right\} _{t\geqslant 1}$ into $$ \underbrace{Z_1,\cdots ,Z_T}_{H_1}, \underbrace{Z_{T+1},\cdots ,Z_{2T}}_{H_2}, \cdots , \underbrace{Z_{\left( k-1 \right) T+1},\cdots ,Z_{kT}}_{H_k}, \cdots $$ So $H_1$ means the $i$-th block in $\left\{ Z_t \right\} _{t\geqslant 1}$. We only sliced the process in Q1, so the conclusion of Q1 remains unchanged.
(Q2, Infinite Case) Now let $N = \infty$ and $T < \infty$. According to "cut" method above, suppose $H_i$ satisfies the following properties:
- The initial state of $H_i$ is $Z_{\left( i-1 \right) T+1}$. It is i.i.d. with $Z_{\left( i-1 \right) T}$, the ternimal state of $H_{i-1}$. Denote $Z_{\left( i-1 \right) T+1}, Z_{\left( i-1 \right) T}\sim \mu _i$.
- Moreover, $\mathcal{F} _{\left( i-1 \right) T+1: iT}=\sigma \left( H_i \right) $ is independent of $\mathcal{F} _{1:\left( i-1 \right) T}=\sigma \left( \left\{ H_1,\cdots ,H_{i-1} \right\} \right) $.
- Process in $H_i$ must be a truncation of a $\beta$-mixing markov process, i.e. $\left\{ Z_{\left( i-1 \right) T+t} \right\} _{t\geqslant 1}$ is a $\beta$-mixing markov process with stationary distribution $\nu _i$.
Hint: So you cannot use any general Asymptotic Properties directly for markov process.
- There exists limit distribution $\nu $ s.t. $D_{\mathrm{TV}}\left( \nu _i, \nu \right) \rightarrow 0$, and $D_{\mathrm{TV}}\left( \nu _{i-1}, \nu _i \right) \rightarrow 0$ as $i \rightarrow \infty$. Also, suppose $D_{\mathrm{TV}}\left( \mu _i, \nu _{i-1} \right) \rightarrow 0$. The divergence $D_{\mathrm{TV}}$ means total variation distance.
Hint: This condition is almost the most important, as it implies the relationship between random processes under different blocks. Although the partitioning of each process is independent, we suggest that over time, $H_i$ progresses towards a strictly stationary process, and the initial distribution of the next block is very close to the stationary distribution of the previous block.
Question: Can we claim that $\left\{ Z_t \right\} _{t\geqslant 1}$ (note it is not markovian) is $\beta$-mixing? Why?
Additional Question: If the answer is not, Can we achieve our final goal by fixing the proposition 4 above? I.e. convergence in probability or a.s. or ...
Here is the counter-example from my comments (simplified to take $N=1$):
Fix $N=1$.
Define $\{Z_2, Z_3, Z_4, ...\}$ as a 3-state ergodic DTMC with state space $\{0,1,2\}$ and steady state distribution that is equally likely over all three states.
Assume $Z_2=0$ surely (so $Z_2$ is independent of everything).
Define $X_1=\sum_{i=2}^{\infty} Z_i10^{-i}$.
Then $X_1$ and $Z_2$ are independent but the process $(X_1, Z_2,Z_3, ...)$ does not have the conditional mixing property because knowing $X_1$ also gives you the full future $Z_i$ values.
You can get the mixing you want if you just concatenate two independent processes $(X_1, ..., X_N)$ and $(Z_{N+1}, Z_{N+2}, ...)$ to create $(X_1, ..., X_N, Z_{N+1}, Z_{N+2}, ...)$. The reason the above counter-example works is that, just because $(X_1, ..., X_N)$ is independent of $Z_{N+1}$ does not mean $(X_1, ..., X_N)$ is independent of $Z_{N+2}$.