How come the proportion of heads to tails across a large number of coin flips tend toward $1:1$ if all outcomes are equally likely?

Question

I can't wrap my head around why the Law of Large Numbers applies in the case of coin flips (in this case that the proportion of heads to tails converges to $1:1$ over time). This is because, as I understand it, every permutation of $H$ and $T$ for $n$ flips is supposed to be equally likely, for example, $HTHTHTHT$ is just as likely as $HHHHHHHH$. My main suspicion is that perhaps the outcomes that have a more balanced proportion of heads to tails are more likely (since there are more balanced outcomes than extreme ones in every set of possible outcomes, or at least so I've seen). But even then, I also know that the probability of getting an equal number of heads and tail decreases as the number of coin flips increases, so I end up getting confused. I appreciate any ideas you can share.

Answer 1

Comments by @MikeEarnest and @saulspatz make good points about the increasing number of possibilities as $n$ increases. Perhaps a visualization of what happens with increasing $n$ will help you get an intuitive idea what is going on. There really are lots of possibile sequences of Heads and Tails, but they have to obey the rules of probability and the proportion of Heads must 'converge' toward 1/2.

Let $S_n \sim \mathsf{Binom}(n, 1/2),$ the number of Heads seen in $n$ tosses of a fair coin. The usual estimate of $p = 1/2$ is the proportion of Heads observed $\hat p_n = S_n/n.$

Then a statement in the proof of the Law of Large Numbers (LLN) says that the probability $$Q_n = P\{|\hat p_n - 1/2| < \epsilon\} > 1 - \frac{1}{4n\epsilon^2}.$$ In other words for any small value $\epsilon > 0,$ the error of the estimate $\hat p_n$ of $p = 1/2$ becomes increasingly likely to be smaller than $\epsilon.$

But in the binomial case, we can actually compute the exact probability $Q_n,$ whereas the LLN just gives a bound that suggests the probability of discrepancies smaller than $\epsilon$ approaches $1.$

B = 800;  n = 1:B;  eps = .05;  dlt = .04/B  # dlt to avoid equality in CDF
Q = pbinom(n/2 + n*eps - dlt, n, .5) - pbinom(n/2 -n*eps, n, .5)
plot(n, Q, type="l", xaxs="i");  abline(h=0:1, col="green2")
  curve(1-1/(4*x*eps^2), 1,1000, lwd=2, col="red", add=T)

Below, the jagged black line shows exact probabilities that $|\hat p_n - 1/2| < .05$ are increasing as $n$ increases, while the red curve from the LLN gives a rather loose bound on the probability of such a discrepancy. (The red line goes up to $1$, but off the graph to the right.)

Answer 2

There are two things at work here. It is true that all sequences of $2n$ flips are equally probable, so if you pick a specific sequence that has $n$ heads and $n$ tails it is the same chance to get that as all heads. However, there are lots of sequences that have $n$ heads, so the chance of getting exactly $n$ heads is ${2n \choose n}$ times more than the chance of getting all heads.

As you say, the chance of getting exactly $n$ heads out of $2n$ flips decreases as $n$ increases. The central binomial coefficient, ${2n \choose n} \approx \frac {4^n}{\sqrt {\pi n}}$ so the chance of getting exactly $n$ heads is about $\frac 1{\sqrt{\pi n}}$ which decreases as $n$ increases. What increases as $n$ increases is the chance of being within a fraction $x$ of half. If you want the chance of being within $0.01n$ of even, that increases as $n$ increases. In the normal approximation the fractional standard deviation is proportional to $\frac 1{\sqrt n}$ so more and more of the peak is within any constant fraction of $n$.