If I am understanding, the law of large numbers correctly, one implication is this:
Let $\lambda$ be the Lebesgue measure on $[0,1]$. Let $f_1,f_2,\dots,$ be functions, $[0,1] \rightarrow \mathbb{R}$, such that
- (Identically distributed) For all $x \in \mathbb{R}$, for all $i,j \in \mathbb{N}$, \begin{align*} \lambda(\{\omega \in [0,1]: f_i(\omega) \leq x\}) = \lambda(\{\omega \in [0,1]: f_j(\omega) \leq x\}) \end{align*}
- (Independent) For all $x_1, \dots, x_k \in \mathbb{R}$, for all $\{i_1,\dots, i_k\} \subseteq \mathbb{N}$\begin{align*} &\lambda(\{\omega \in [0,1]: f_{i_1}(\omega) \leq x_1, \dots, f_{i_k}(\omega) \leq x_k\}) \\ &= \lambda(\{\omega \in [0,1]: f_{i_1}(\omega) \leq x_1\})\cdots\lambda(\{\omega \in [0,1]: f_{i_k}(\omega) \leq x_k\}) \end{align*}
- (Integrable) $\int_0^1 f_1(\omega)d\lambda(\omega)$ exists and is finite.
Then there exists $A \subseteq [0,1]$ with $\lambda(A) = 1$ such that for all $\omega\in A$, \begin{align*} \lim_{n\rightarrow \infty} \frac{1}{n}\sum_{i=1}^n f_i(\omega) = \int_0^1 f_1(\omega)d\lambda(\omega) \end{align*}
My question is: if we weaken the requirement that $f_1$ is Lebesgue integrable, and instead merely require that it is Henstock–Kurzweil integrable, does this theorem still hold? That is, does $\frac{1}{n}\sum_{i=1}^n f_i(\omega)$ converge to the Henstock-Kurzweil integral?
I ask this because if this theorem still holds, then that suggests that the Henstock–Kurzweil integral ought to be used instead of the Lebesgue integral, as we might encounter, for example, a statistical experiment where we have iid random variables that are not Lebesgue integrable, but are Henstock–Kurzweil integrable. In this situation, if we only considered Lebesgue integrability, we would conclude that the sample average does not converge, whereas in reality, it would, to the Henstock–Kurzweil integral.
Likewise, if it does not hold, then that suggests that the Lebesgue integral ought to be favored in the context of probability; otherwise we may mistakenly conclude that the sample average converges, when it does not.
In addition, this further raises the question of whether the (proper) Riemann integral may be superior to the Lebesgue integral in some situations. It seems to me that this is never the case in the context of probability, but perhaps it may be the case in, say, kinematics?