In the book I'm currently reading, this law is stated as:
The Weak Law of Large Numbers provides proof of the notion that if $n$ independent and identically distributed random variables, $X_1,X_2,...,X_n$, from a distribution with finite variance are observed, then the sample mean, $\bar{X}$, should be very close to $\mu$ provided $n$ is large.
The problem is, the book hasn't formally defined $\bar{X}$, so I'm unclear as the the relationship between $X$ and the $X_i's$. From the context I assuming that:
$$\bar{X} = \frac{X_1+X_2+\cdots+X_n}{n}$$
But this makes no sense to me since the $X_i's$ may not necessarily all even be the same unit, so this would be like adding apples and oranges. In other words, I can say, in an experiment of three coin tosses, let $X_1$ be the number of tails that appear and $X_2$ be the number of heads that appear, thus the units for $X_1$ and $X_2$ are tails and heads, respectively.
Understanding the relationship between the $X_i's$ and $X$ is important to me to make sense of the expression:
$$\lim_{n \rightarrow \infty} \mathbb P \left( \left| \frac{X_1+X_2+\cdots+X_n}{n}-\mu \right| \ge \epsilon \right) = 0$$
which, upon replacing $\mu$ with its definition, I obtain:
$$\lim_{n \rightarrow \infty} \mathbb P \left( \left| \frac{X_1+X_2+\cdots+X_n}{n}-E[X] \right| \ge \epsilon \right) = 0$$
Perhaps this is totally wrong but my interpretation of this law is essentially if $\mu_i=E[X_i]$, then, as $n \rightarrow \infty$, $\mu_1=\mu_2=\cdots=\mu_n=\mu$, which I feel is intuitively obvious as all the $X$'s are identically distributed (again, ignoring units).
So my questions boil down to:
- What is the definition of $\bar{X}$?
- What is the conceptual meaning of $\frac{X_1+X_2+\cdots+X_n}{n}$?
- How do I reconcile the unit differences of the expression in question 2?
- What is the relationship between the $X_i$'s and $X$?
Your definition of $\bar{X}$ is correct. I think you've just chosen a poor example to try to apply it to.
In your first example, if $X_1$ is number of heads and $X_2$ is number of tails, I would argue that both values have units of "coins", so it does make sense to add them. $X_1 + X_2$ is the total number of coins which are showing either heads or tails (which is always 3). But they are not independent so the weak law of large number will not have anything to say about this sum.
For a better example, suppose you roll $n$ dice, and $X_i$ is the number of pips showing on the $i$th die. These all have the same units (pips) and $X_1 + \dots + X_n$ is the total number of pips showing. On different runs of the experiment, it could be anywhere between $n$ and $6n$. Then $\bar{X} = \frac{X_1 + \dots + X_n}{n}$ is the (empirical) average number of pips per die. On different runs of the experiment, it could be anywhere between 1 and $n$.
The weak law of large numbers says that when $n$ is very large, this observed value is unlikely to be very far from its theoretical mean of $3.5$.