Time vs. Ensemble Distribution of Sample Mean Z-Scores

Question

Suppose we have a sequence of bounded i.i.d variables $\lbrace X_n \rbrace_{n \in \mathbb{N}}$ with unit variance and zero mean. Let $\bar X_n$ denote the sample mean of the first $n$ variables in the sequence. From the central limit theorem we have that,

$$Z_n = \sqrt{n} \cdot \bar X_n \to_{\mathcal{D}} \mathcal{N}(0,1)$$

What about the uniform distributions over the elements $S_n = \lbrace Z_k \rbrace_{k \le n}$ which are the set of the Z-scores up to a certain time? Effectively, I'm wondering about the long-run distribution of the Z-scores.

At first, I thought this might also converge to the standard normal, but this doesn't seem to be the case. See the plots below where I show the histogram associated with the sequence of Z-scores. From this, I suspect that the time average might actually be a random limiting distribution. What's the actual situation? Is there a way to characterize the long-term behavior of the time average for the Z-scores? My guess is that the distribution for,

$$h_{a,b} \sim \lim_{T \to \infty} \frac{1}{T} | \lbrace k \in [0,T] : S_k / \sqrt{k} \in [a,b] \rbrace |$$

has singularities in its PDF at the points $\lbrace 0, 1 \rbrace$ since this is the case for $h_{0,\infty}$ which is arcsine distributed.

Answer 1

The distribution of the long run average of the Z transforms (Z scores) is exactly the same as that of the original variable up to a parameter change.

Explicitly, if the original averages converges in probability to a normal distribution with a given mean and standard deviation, the average of the Z transforms will converge to a normal distribution with expectation 0 and variance 1.

To see this, it is necessary to write down what it means for the average to converge in distribution to a normal variable. This means that the limit of the probability that the average will be below a given value converges to the cumulative distribution function of a normal with a certain mean and variance. If you take the limit in probability of the average of the Z transforms being below a given value $X$, this will be equivalent to the average being below $\sigma X + \mu$. This in turn implies that the average of the Z transforms converges in distribution to the same distribution as the original averages, up to a rescaling of the mean and standard deviation.

Answer 2

We can show that there is no stationary distribution for the $Z_n$ as defined in your question. Consider a simple quadrature $\mathcal{Q} = ((-\infty, 0], (0, \infty))$. Then the liklihood of seeing a particular histogram (as characterized by positive sojourn time) is given by the distribution of,

$$H(0, \infty) = \frac{1}{T} | \lbrace k \in [0,T] : S_k > 0 \rbrace |$$

This random variable follows an arcsine distribution. This is also not hard to show empirically.

A process with an ergodic limit obeys the law of large numbers and would have an empirical function that looks as though it was converging to a Dirac delta function. From this, we conclude that the z-scores do not 'mix' well.

Some last comments. Paul Levy first formulated the arcsine law used above. The style of argument presented above is loosely related to local-times, another of Levy's contributions and a powerful device used in the analysis of integrated stochastic processes. Here is a somewhat accessible simple introduction. The arcsine appears elsewhere. Erdos showed the distribution of prime divisors satisfies an arcsine law.