Posterior, prior, likelihood exercise

Question

I'm trying to solve a, what is supposed to be, a simple exercise regarding finding posterior.

Exercise is following:

Assume you are making repeated independent experiments with a probability of success θ in each experiment. Initially, you make 12 experiments, of which 9 are successful.

a) Using a prior for $\theta$ that is uniform on the interval $[0, 1]$, what is the posterior for $\theta$ given the results of the 12 experiments?

I'm new to this, but my solution so far is:

• I know that posterior $\propto$ likelihood $\times$ prior, that is $\pi (\theta | data) \propto \pi (data | \theta) \pi(\theta)$

• I've not completely understood posterior, likelihood and prior in detail. But I do know that Beta and Binomial belongs together, for which I also noticed that $\theta \sim unif[0,1]$ is equivalent to $\theta \sim Beta(1,1)$ and since we are dealing with experiments with two possible outcomes, I think of Binomial distribution.

• With these things noticed I don't know how to proceed with the calculation of $\pi (data | \theta) \pi(\theta)$

Would anyone like to help me out?

Answer 1

As you mentioned in the post and the comments, here are the prior and likelihood functions:

• $\pi(\theta) = 1$ (on the interval $(0, 1)$).
• $\pi(\mathrm{data}\mid\theta) = \binom{12}{9}\theta^9(1 - \theta)^{12 - 9}$.

Now if you multiply them, you should get (up to a multiplicative constant) the posterior distribution:

$$\pi(\theta \mid \mathrm{data}) \propto 1 \cdot\binom{12}{9}\theta^9(1 - \theta)^{12 - 9} \propto \theta^9(1 - \theta)^3.$$

Remember, this is a distribution for the probability of success $\theta$, so it is supported on the interval $(0, 1)$.

The question is now: can you think of a continuous distribution on the interval $(0, 1)$ such that its density function, up to a multiplicative constant, looks like the posterior expression derived above?

Alternatively, you can take the expression $\theta^9(1 - \theta)^3$ and integrate it from 0 to 1. This will tell you what the multiplicative constant should be.

---

As mentioned in the comments, the expression $\theta^9(1 - \theta)^3$ looks a lot like the PDF of the Beta distribution with $a=10$ and $b=4$. And in fact, since we know that the posterior PDF has to integrate to 1, we can conclude that the posterior is indeed $Beta(10, 4)$.

An important aside: the fact that we get $\pi(\theta \mid \mathrm{data}) \propto \pi(\mathrm{data}\mid\theta) \pi(\theta)$ and not an equality is one of the main reasons why Bayesian statistics is computationally difficult. We only know the posterior distribution up to a constant, and we often need to learn about this distribution without ever computing the constant.