In my probability class, we covered the proof of the following result, known as the "strong law of large numbers":
Theorem. Let $(\Omega,\mathscr F,P)$ be a probability space, $\{X_n\}_{n\in\mathbb N}$ be a sequence of i.i.d. random variables with finite mean $E[X_n]=\mu<\infty$, and for every $n\in\mathbb N$, let $S_n=X_1+X_2+\cdots+ X_n$. Then, $S_n/n$ converges to $\mu$ with probability $1$.
As the professor was writing the result down, he mentioned that an important application of this theorem in statistics is that it shows that the empirical mean is a consistent estimator of the theoretical mean of random variables with finite mean. Thinking about this lead me to the following question:
Question. In statistics, what is the mathematical model for a finite collection of "independent trials"? More precisely, suppose we have a random variable $X:\Omega\to\mathbb R$ with a certain distribution $F:\mathbb R\to[0,1]$, how does one generate, given $n\in\mathbb N$ a collection $x_1,...,x_n\in\mathbb R$, which can be considered independent trials of that random variable?
My attempt at an answer so far:
Attempted Answer 1. One way could be to select $n$ elements in the sample space $\omega_1,...,\omega_n\in\Omega$ and consider the collection $X(\omega_1),...,X(\omega_n)$. After all, $X(\omega)$ is seen as a "realization" of the random variable $X$, and so we would obtain $n$ realizations. However, it is unclear how we could make these realizations independent.
Attempted Answer 2. Another idea would be to define a collection of i.i.d. random variables $X_1,...,X_n$ and then evaluate them at an arbitrary $\omega\in\Omega$, which would give the collection of trials $X_1(\omega),...,X_n(\omega)$. In this case, the independence is not a problem, but the random variables $X_i$ themselves seem cryptic: where do they come from? How do we define them from $X$?
You need to use one big probability space. For $n$ i.i.d random variables, that probability space has the sample space $\mathbb{R}^n$ (assuming that random variables are real-valued), and the measure is simply the product measure $\underbrace{\mathbb{P}\times\ldots\times\mathbb{P}}_{n\text{ times}}$. $\mathbb{P}$ is the probability measure corresponding to a single random variable.
The same works for independent, but not necessarily identical distributed random variables, you instead use the product measure $\mathbb{P}_1\times\ldots\times\mathbb{P}_n$, where $\mathbb{P}_i$ is the probability measure corresponding to the $i$-th random variable.