Does it make sense to ask what the probability that a coin is biased (in favour of heads) is?

Question

A coin is tossed $5$ times and $5$ heads are observed. What is the probability the coin is biased in favour of heads? Does this question even have any meaning without using a significance level?

Let's have a look at the usual Binomial test.

Let $X$ be the random variable: the number of heads out of $5$ tosses. Then $X \sim B(5,p).\ H_0: p = 0.5;\ H_1: p > 0.5.$

Suppose $H_0$ is true, that is, $X \sim B(5,0.5)$. Then $P( X \geq 5) = P(X=5) = 0.5^5 = 0.03125.$ This is the probability that the coin actually is fair, and we just happened to get $5$ heads in a row. My question is: is the probability that the coin is not biased in favour of heads equal to $0.03125$? Or like I said at the beginning, is this question meaningless?

Answer 1

Yes, this question makes sense. However, to answer it you need first a prior distribution for the probability of rolling heads (i.e. your initial suppositions on possible values of $p$ presented in form of probability distribution). Because you decide between $p = \frac{1}{2}$ and $p > \frac{1}{2}$ (no matter how much), I, personally, think that the following prior will be most natural in this case: $p$ equals to $\frac{1}{2}$ with probability $\frac{1}{2}$ and is uniformly distributed on $[\frac{1}{2};1]$ otherwise.

Now, you need to find the posterior probability the following way (using Bayes theorem):

$$P(p=\frac{1}{2}|\text{ 5 heads in 5 tosses}) = \frac{P(\text{ 5 heads in 5 tosses}|p=\frac{1}{2})P(p=\frac{1}{2})}{P(\text{ 5 heads in 5 tosses})}$$

Note, that $P(\text{ 5 heads in 5 tosses}|p=p_0) = p_0^5$ for any $p_0 \in [\frac{1}{2}; 1]$ and thus for our prior we have:

$$P(\text{ 5 heads in 5 tosses}) = \frac{1}{2}(P(\text{ 5 heads in 5 tosses}|p=\frac{1}{2}) + \int_{\frac{1}{2}}^{1} P(\text{ 5 heads in 5 tosses}|p=t)dt) = \frac{1}{2}(\frac{1}{32} + \int_{\frac{1}{2}}^{1} t^5 dt) = \frac{1}{64} + \frac{1}{12}(1 - \frac{1}{64}) = \frac{25}{256}$$

Thus the posterior probability of the coin being unbiased is $$P(p=\frac{1}{2}|\text{ 5 heads in 5 tosses}) = \frac{\frac{1}{64}}{\frac{25}{256}} = \frac{4}{25}$$

And from that we can conclude, that the posterior probability of the coin being biased towards heads is:

$$P(p>\frac{1}{2}|\text{ 5 heads in 5 tosses}) = 1 - P(p=\frac{1}{2}|\text{ 5 heads in 5 tosses}) = \frac{21}{25}$$

Note, that you can get a different result from different prior (but the procedure of obtaining it will still be the same).

Answer 2

Joining the two reasoning, Classical and Bayesian, I suggest to assume an uniform prior in $(0;1)$ thus we can get focus on the likelihood (I think this is better for the O.P as he said he didn't study Bayesian Inference yet.

The observed likelihood (5 H on 5 Tosses) is the following

$$L(\theta) =\theta^5$$

If we look at this function as a function of $\theta$ and we want to consider it as a density, we have to normalize it setting

$$C\int_0^1 \theta^5 d \theta=1$$

finding $C=6$

Thus we can use the following density (whithout explicitly introducing Beta distribution)

$$f_{\Theta}(\theta)=6\theta^5$$

$\theta \in (0;1)$

...and now we can compute the probability that the coin is biased in favour of head in the following way

$$\int_{0.5}^{1}6\theta^5 d \theta=\frac{63}{64}$$

Graphically, the probability the coin is biased in favour of H is the purple area below

---

@Adam Rubinson: For a basic review of Bayesian thinking, if interested, you can read this paper