A basic intuitive confusion on coin tosses

Question

I have a very basic question on independent coin toss with probability of head $p$. Suppose, we do $n$ coin tosses. Then we can expect on an average $np$ heads will be produced (although it is actually true when $n \to \infty$). Now, consider this scenario with first $n-1$ tosses producing all tails. Then will the probability of head coming on the $n$-th trial is very high ? If yes, then the coin tosses will be dependent.

Answer 1

No. This is a fundamental mistake that many of the public make in interpreting probability. Clearly these are independent events- just consider a real coin or a real die etc. When I pick up a coin, it has no memory of its previous flips and how many were heads or tails. On average, yes, we'd expect half the tosses to be heads and half to be tails but this doesn't have to be the case (rather there is a binomial distribution involved). The case where you've already seen the outcome of several tosses can essentially be thought of as an example of conditional probability and if you interpret it in this manner, you will see that the outcome of each individual toss has exactly the same probability regardless of previous outcomes of tosses.

Answer 2

The probability of the $n$-th coin toss producing head is exactly $p$, even if the previous ten million tosses were tails.

However if a coin toss gave ten million tails in a row, then unless $p$ is very close to $0$, I'd strongly doubt that the probability of the coin to produce heads really is $p$. That is, in reality many tails in a row would make you give a higher probability to tails, but not because the coin had any memory, but because you would doubt that the information you've been given, namely that the probability of heads is $p$, is accurate (how many tails it needs to cause you to doubt of course depends on as how reliable you consider the source of the information).

But there's absolutely no reason to assume that heads get more probable the more tails have been tossed.

Also note that the probability of the coin producing exactly $np$ heads goes to zero, even for those values of $n$ where $np$ is integer. Indeed, the absolute deviation from that value will grow like $\sqrt{n}$. However the probability that the frequency lies in an arbitrary small interval around $p$ goes to $1$ for $n\to\infty$.