Finding probability of a sample using Bayesian Inference

Question

In a particular water sample, ten bacteria are found, of which three are of type A.

What is the probability of obtaining six type A bacteria, in a second independent water sample containing 12 bacteria in total?

Answer 1

I'm surprised this has not been answered yet, so I'll have a go. Let $\theta$ be the proportion of type $A$ bacteria in the population, and suppose the samples are independent with replacement, so that the number of type $A$ bacteria in a sample follow the binomial distribution.

The answer is going to depend on your prior on $\theta$. Let's say the prior is $p(\theta)$. The likelihood (the probability of getting three successes in $10$ trials) is proportional to

$$\theta^3 (1-\theta)^7$$

so by Bayes' Theorem, the posterior distribution for $\theta$ is proportional to

$$\theta^3 (1-\theta)^7 p(\theta)$$

Usually, the prior $p(\theta)$ is chosen to make this tractable. Let us suppose that $p(\theta)$ is flat, ie. $p(\theta) = 1$. Then the posterior has the form $\theta^{a-1}(1-\theta)^{b-1}$, you can see that $\theta$ has a $\mathrm{Beta}(4,8)$ distribution. So the posterior for $\theta$ is

$$f(\theta|\mathrm{data}) = \frac{1}{B(4,8)}\theta^3 (1-\theta)^7.$$

The probability of getting six successes in a new set of $12$ trials is given by

$$\int_0^1 f(\theta|\mathrm{data}) {12 \choose 6} \theta^6(1-\theta)^6 d\theta$$

which simplifies to

$${12 \choose 6}\frac{1}{B(4,8)}\int_0^1 \theta^9 (1-\theta)^{13} d\theta = {12 \choose 6}\frac{B(10,14)}{B(4,8)} \approx 0.107.$$

If you had chosen a different prior, you would have got a different answer. For example, if you choose a $\mathrm{Beta}(0.5,0.5)$ prior then you end up with

$${12 \choose 6}\frac{B(9.5,13.5)}{B(3.5,7.5)} \approx 0.099.$$

By choosing various different priors, you could get any answer between $0$ and $0.226$ (the probability of getting $6$ successes out of $12$ trials when the success probability is $0.5$.) An answer of about $0.1$ looks sensible.