Bernoulli's Theorem says the following: If S represents the number of successes obtained during $n$ Bernoulli trials, and if $p$ is the probability of success, then $$\dfrac{S}{n} \to p \text{ as } n \to \infty.$$
I am wondering if this can be generalized to a sequence of iid random variables that are not necessarily Bernoulli. Here is what I have in mind:
Suppose that $X_1, X_2, \dots$ is a sequence of iid random variables. Then for almost all $\omega$, we have: $$\lim_{N \to \infty}\dfrac{\# \{n < N: X_n(\omega)=a \}}{N} = \text{Pr}(X_1=a).$$ In other words, the relative frequency of an event should approach the theoretical probability of that event as the number of trials goes to infinity. (I picked $X_1$ on the RHS but I could have picked any $X_i$ because the random variables are iid.)
My question is: Is this theorem true? If so, does anybody have a reference to a proof?
Since $Y_n = 1_{\{X_n = a\}}$ are i.i.d., that's a special case of the strong law of large numbers (https://en.wikipedia.org/wiki/Law_of_large_numbers). In the strong law of large numbers, you get almost sure convergence, and if $E(|X_1|^p) < \infty$ you get convergence in $L^p$. See Klenke's probability book or Durrett's book for several different proofs (direct proof, martingale proof, ergodic theory proof, etc.).