Proof that as a sample size grows, its mean gets closer to the average of the whole population

66 Views Asked by At

I think it is common knowledge that as a sample size grows, the probability of its mean, $\overline{X}_n$, getting closer to the average of the whole population, $\mu$, increases, i.e., \begin{equation}\tag{1}\label{eq:desire} \forall \epsilon \in {\rm I\!R},\, n_1 > n_2 \implies \rm{P}(|\overline{X}_{n_1} - \mu| < \epsilon) \leq \rm{P}(|\overline{X}_{n_2} - \mu| < \epsilon) \end{equation} where \begin{equation} \overline{X}_{n}={\frac {1}{n}}(X_{1}+\cdots +X_{n}) \;, \end{equation}

I have seen this concept being justified with the law of large numbers (LLN) but, to my understanding, the convergence specified by the (strong) law of large numbers only characterizes $\lim_{n\to\infty} \overline{X}_n$: \begin{equation} \rm{P}\big( \lim_{n\to\infty} \overline{X}_n = \mu \big) = 1 \end{equation} And does not specify anything about finite numbers of observations.

So, my question is can \eqref{eq:desire} be derived (e.g. from the LLN or the CLT) or is there any law that states something similar to it?