I imagine this question has already been asked, but I haven't found it yet.
Suppose there is a multiple choice question. Some students know the answer, and others do not. If a student knows the answer they always get the question right. If a student does not know they guess randomly, there are four answer choices.
Suppose the probability of a student knowing the answer is $\pi$. You only observe the selected answers. How would you go about finding a $95\%$ confidence interval for $\pi$?
MY ATTEMPT
I let $X_i$ be an indicator variable that is observed. $X_i=1$ if the student gets the correct answer, and $X_i=0$ if not. I let $Z_i$ be an indicator variable for if a student knows the answer (0=does not know, 1=knows).
$$X_i=1 |Z_i = 0\sim Bern \left (\frac{1}{4}\right )$$ $$P(X_i=1 |Z_i = 1) = 1$$
Therefore,
$$P(X_i = x_i) = \left ( .25\right )^{x_i}\left ( .75\right )^{1-x_i}(1-\pi) + \pi$$
I am at a loss on how to make this into something more friendly or familiar to create a CI. Although I can see a few ways to do this computationationally, that is not what I am going for.
Let's say the chance of answering a question correctly is $p$. From the conditions of the problem, $$p = \pi + (1 - \pi)\frac{1}{4} \tag{*}$$ From the observed answers, you can get a $95\%$ confidence interval on $p$ using the standard formula for a confidence interval on proportions. This gives you limits on $p$, say $p_a < p < p_b$. Now you can go back to (*) and solve for $\pi$ in terms of $p$ to find the corresponding limits on $\pi$.