I have a survey of a large number of questions. Each question is multiple choice, and has three possible answers.
Users get served random questions to answer. So they do not all answer the same set of questions.
If "user 1" and "user 2" have both answered 'm' (say 90) questions in common, and of these 'm' questions, 'n' have been the same answer (say 70).
If my null hypothesis is "user 1 and user 2 have uncorrelated answers", how would I create a t-state from the above to reject / not reject.
Thank you!
We would expect them to have $\frac{m}{3}$ answers the same assuming their answers are random. Letting $\hat{p}=\frac{n}{m}$ denote the proportion of similar answers observed and $p=\frac{1}{3}$, let
$$H_0 : p = \frac{1}{3}$$
$$H_a : p \neq \frac{1}{3}$$
From here, this is just a simple one sample test for proportions where $$Z=\frac{\hat{p}-\frac{1}{3}}{\sqrt{\frac{\left(\frac{1}{3}\right)\left(\frac{2}{3}\right)}{m}}}\sim\mathsf{N}(0,1)$$