A priori probability in Bayesian inference problem

Question

The problem

A psychic uses a five-card deck to demonstrate ESP, claiming to be able to guess a card correctly with $0.5$ probability (of course, ordinary guessing is $0.2$). A single experiment consists of making five guesses, reshuffling the deck after each guess. The experiment is executed and the the psychic guess correctly $3$ out of $5$ times. Assuming that the only two possibilities are "ESP" and "not-ESP" how high must the a priori probability that the psychic really has ESP be, in order for the a posteriori probability that the psychic has ESP to be at least $0.7$?

Thoughts

Let

$X = \#$ of correct guess after $5$ attempts.

My understanding is that the prior distribution has to be a function of a parameter $\theta$

$\theta = $ probability that the psychic has SP

Q1: Is this the correct parameter or should I choose a different one?

Assuming that this is the case, I need to choose a distribution for $\theta$.

Q2: Is the uniform distribution $Unif(0,1)$ an appropriate distribution?

My understanding is that $f(x|\theta) \sim Bin(5, \theta)$ (but please correct me if there is motivation for a better choice for the distribution).

Putting all this together:

$$ f(\theta | X = 3) \propto f(X = 3 | \theta) f(\theta) $$

I am not sure, though, how all this is supposed to help me find the desired probability. I think that part of the confusion is from the terminology a prior and a posteriori probability in this particular context.

I would appreciate any hints, suggestions, clarifications about this problem. Thank you very much in advance.

Answer 1

The way you have written it, "$\theta$ = probability that the psychic has ESP", $\theta$ essentially is your prior distribution. There are only two possibilities, ESP and not-ESP, so the full statement of the prior is (I write $e$ for ESP and $\neg$ for negation):

• $P(e) = \theta$
• $P(\neg e) = 1 - \theta$

Writing $d$ for the observed data (3 out of 5 guesses correct), the inference is: $$ P(e\mid d) = \frac{P(e) P(d\mid e)}{P(d)} $$

I take "ordinary guessing" (independently at $0.2$) to mean $P(d\mid\neg e)$, and the denominator $P(d)$ has to be taken as $P(d\wedge e) + P(d\wedge\neg e)$, which can be expressed in terms of your unknown and the given binomial probabilities as follows: $$ P(d) = P(d\mid e)P(e) + P(d\mid \neg e)(1 - P(e)) $$ The only unknown is $\theta = P(e)$. Solve the inequality $P(e\mid d)\geq 0.7$.

("Prior" means just "prior to data" or "prior to this particular experiment". The question here is: how inclined you need to be to believe that this psychic is genuine, before this experiment, for this experiment to convince you at least that much.)

Answer 2

First, some basic calculations. let $p$ be the probability of guessing a card correctly. Then the probability of getting exactly $3$ correct is $\binom 53 p^3(1-p)^2$. If $p=.2$ this is $.0512$, if $p=.5$ this is $.3125$

Let's say your prior is $\theta_0$. That is, before you test anything, you estimate that the "ESP probability" is $\theta_0$. Then, amongst your pre-experiment sample space, you will see three out of five in $.3125\theta_0 +.0512(1-\theta_0)$ of all possible experiments. But then our new estimate is $$\theta_1=\frac {.3125\theta_0}{.3125\theta_0 +.0512(1-\theta_0)}$$

Setting this to $.7$ and solving we see that we need $\theta_0≥\frac {3584}{12959}\sim .2766$