How to construct a simple test to signficance level of $5\%$

Question

I am stuck at the following homework question:

You visit a fortune teller. You test his abilities by tossing a fair coin 100 times. The fortune teller manages to predict the outcome of 60 tosses correctly. Construct a statistical test for the hypothesis $$H_0: \text{The fortune teller is a fraud.}$$ Does your test reject the hypothesis with a significance level of $5\%$?

I would say that the obvious idea would be to define the critical area $K$ of the test as follows:

$$K := \bigg\{(x_1,\ldots,x_n) \in \{0,1\}^n \ \mid \ \sum_{i=1}^n x_i \ge k \bigg\}$$

But how do I know how to choose $k$ to reach the required significance level? Could you help me?

Answer 1

You can run a binomial test considering under the null hypothesis (the fortune teller is a fraud) what is the probability to get by pure guessing $60$ or more correct answers. The proportion expected (the population proportion, so to speak) is $0.5:$

$$\Pr(\text{counts} \geq 60)=\displaystyle \sum_{n=60}^{100} {100\choose n} \,0.5^n\, 0.5^{100-n}$$

or

> pbinom(59,100,.5,lower.tail=F)
[1] 0.02844397

... or given that the sample is large, we can use the normal approximation to test if the sample proportion is different from a population proportion - the equivalent to the $Z$-test for sample means. Under the null hypothesis (random guesses), the probability that the proportion of predictions is $60$ when the expected proportion is $0.5$ is

$$Z=\frac{.6-.5}{\sqrt{\frac{.5(1-.5)}{100}}}=2$$

which is greater than

> qnorm(.95)
[1] 1.644854

Hopefully, though, and despite the evidence, you would use common sense, and not reject the null: the fortune teller is indeed a fraud.