I wonder if someone can explain, in idiot’s terms, how you work out the following type of calculation.
We have a group of x independent individuals, each whom are y competent at choosing correctly between binary options, and of whom z have chosen option A over option B. How confident in A should we be?
E.g. we have 10 individuals, who choose correctly .6 of the time, and 7 of whom have chosen option A = how confident should we be that option A is correct?
How do I work out the chance of them having chosen correctly?
Or is this too advanced, and I need to go and do some serious study to have a chance of understanding?
THANKS!
Hmm, I don't know the level of mathematics you're at, so I can't comment whether this is too advanced or not.
Take a simple example, let's start with a single person. Now this person will have a competence level $0 \leq y \leq 1$. You might ask, how confident should we be in this person, given his competence level is $y$? Well if we ask him to answer $n$ questions, he'll answer each of them independently with $y$ level of competence (given those questions were completely different and had nothing to do with each other). So the number of questions that we ask him do not matter, he's gonna answer each with accuracy of $y$ given those questions are independent. In other words, the probability of the question being answered correctly is $y$, and incorrectly is $1 - y$, and hence, our confidence in option A (given that he purposefully chooses option A) being correct is $y$ and it being incorrect is $1 - y$ respectively.
Now let's move onto two persons. Now, we can have four situations:
1) Both choose option A.
2) Both choose option B.
3) and 4) One chooses option A and the other chooses option B.
Given that we don't care about whether person 1 makes the decision before person 2 or not, we can safely say that cases 3 and 4 are identical. Now assuming both of these folks had $y$ level of competence, both would separately answer the question with $y$ level of accuracy. So for each individual case (assuming option A is correct):
1) We have probability $y * y$, as person 1 and person 2 answer with $y$ level of accuracy independently.
2) We have probability $(1 - y) * (1 - y)$, as person 1 and person 2 answer it wrongly independently.
3) and 4) We have probability $y * (1 - y)$ and $(1 - y) * y$, as alternately, one gets it right and other gets it wrong. So for this case, we sum the probabilities of these individual sub-cases as $2 * y * (1 - y)$
So these four cases tell us the individual probabilities of cases where the correct solution is assumed to be A. We must note that even if the correct option isn't A, we will have the same 4 cases. Thus, in order to come up with a value of confidence in A, we will need to choose between one of the above three situations, as it is provided in the question.
You can identify a pattern here for $n$ people. Given that $k$ people out of them choose option A, we have:
Confidence in A: (Choosing $k$ people out of $n$ which have chosen option A) * (Multiply $k$ individual competencies in choosing the correct option) * (Multiply $n - k$ individual incompetencies in choosing the correct option)
Hence the direct answer follows from:
Confidence in A: $\binom n k * y^{k} * (1 - y)^{n - k}$
The above relation follows from combinatorics, I'm not sure if you're aware of it. As a good follow-up, look through Wikipedia articles on Bernoulli and Binomial Distributions. Hope this helps.