Convergence of the Law of large number

Question

I have an "honest" coin and I toss it many times. According to the Law of large numbers, the more I toss the more the head occurrences align to the head probability ($0.5$). Is there a way to measure this convergence?

Because I noticed that, as $N=\text{[number of tosses]}$ grows up, number of heads $\approx $ number of tails, but (number of heads $-$ number of tails) becomes larger.

For example, if $N=100$, it is probable to have $60$ heads and $40$ tails; if $N=1500$, $788$ heads and $712$ tails ---> number of heads better aligns to head probability, but (number of heads $-$ number of tails) becomes larger.

Answer 1

If $H$ is the number of heads and $T$ the number of tails from a fair coin, with $H+T=N$, then

• $H$ has expectation $\frac{N}2$ and variance $\frac{N}4$ and standard deviation $\frac{\sqrt{N}}{2}$, increasing with $N$
• $H-T$ has expectation $0$ and variance ${N}$ and standard deviation $\sqrt{N}$, increasing with $N$
• $\frac{H-T}{H+T}$ has expectation $0$ and variance $\frac{1}{N}$ and standard deviation $\frac{1}{\sqrt{N}}$, decreasing with $N$
• $\frac{H}{H+T}$ has expectation $\frac{1}2$ and variance $\frac{1}{4N}$ and standard deviation $\frac{1}{2\sqrt{N}}$, decreasing with $N$

The Law of Large Numbers is consistent with the last of these

You have correctly observed the increasing/decreasing contrast depending on whether you concentrate on counts or proportions

Answer 2

I noticed that, as N=[number of tosses] grows up number of heads ≈ number of tails, but (number of heads − number of tails) becomes larger.

Yes. The law of large numbers tells you that the sample mean converges (in probability!) to the true mean, that is, that $ \hat p = H/N \to p$, more in detail, that the probability of having $|p - \hat p|>\epsilon$ tends to zero as $N$ grows.

But this does not imply that the difference between tails and heads $|H - T|=| N - 2H|$ tends to zero. (It only tends to zero when divided by $N$).

Is there a way to measure this convergence?

A quick (but not tight) bound is given by Chebyshev's inequality. In our case, it says that

$$P( |\frac{H}{N} - \frac12| \ge \epsilon) \le \frac{1}{4 N \epsilon^2}$$

or, calling $\Delta = T-H$

$$P\left( \frac{|\Delta|}{N} \ge \epsilon\right) \le \frac{1}{ N \epsilon}$$

More (very) precise estimation of the probabability of that deviation can be obtained by the Central Limit theorem.