Fun Inference with Low Number of Data Points.

Question

Apples on my large orchard can either be red or green, I have observed that two apples are red and seen no other apples, what does this tell me about that ratio of red to green apples?

Thanks

Answer 1

The beta family of distributions is commonly used for prior distribution of the binomial success probability $\theta$. The first reason is that $(0,1)$ is the support for beta distributions. The second is that the beta family has many different shapes. The third is that beta priors are 'conjugate' to (mathematically compatible with) binomial likelihoods, thus making distributional computations easy.

Let Success be getting a red apple on a random draw from your orchard and suppose your prior density is $p(\theta)$ is $Beta(3,3),$ which means you believe the ratio may be 'somewhere near' 50:50, but not necessarily 'very close'. In particular $P(.2 < \theta <.8) \approx 0.88.$ Also, as we begin a 90% Bayesian probability interval for $\theta$ is $(.189, .811).$ In R:

 diff(pbeta(c(.2,.8), 3, 3))
 ## 0.88416
 qbeta(c(.05,.95), 3, 3)
 ## 0.1892554 0.8107446

Then your binomial likelihood function based on 2 successes out of 2 is $$p(x|\theta) \propto \theta^2 (1 - \theta)^{2-2} = \theta^2,$$ where the symbol $\propto$ indicates that we have omitted the unnecessary binomial coefficient.

Then Bayes' Theorem says $$\text{POSTERIOR} \propto \text{PRIOR} \times \text{LIKELIHOOD}$$ or $$p(\theta|x) \propto p(\theta)p(x|\theta) \propto \theta^{3-1}(1 - \theta)^{3-1} \times \theta^2 = \theta^{5-1}(1-\theta)^{3-1},$$ where we recognize at the right, the kernel of $Beta(5,3).$

Thus, our posterior 90% probability interval for $\theta$ is $(.341, .871)$---slightly more optimistic about getting red apples from your orchard than before, but then two observed red apples don't count for a lot if your prior is even mildly concentrated at '50:50-ish',

 qbeta(c(.05,.95), 5, 3)
 ## 0.3412614 0.8712436

Note: If we had begun with a noninformative prior $Beta(.5, .5)$ our 90% interval would have gone from prior $(.006, .994)$ to posterior$(.431, .999)$.

 qbeta(c(.05,.95), .5, .5)
 ## 0.00615583 0.99384417
 qbeta(c(.05,.95), 2.5, .5)
 ## 0.4307415 0.9991318

When we have very little data, the prior distribution makes a big difference in the posterior. So, indeed as suggested by @algamest, it really does depend on your prior distribution. However, an advantage of applied Bayesian statistics is that yesterday's posterior can be today's prior. So you can refine your opinion about the proportion of red apples in your orchard as you continue to (randomly) collect and munch apples over time.