Suppose I have a three-way election between candidates A, B, and C, with 1,000,000 voters.
A national poll on a fair sample of N=1000 yields the following breakdown, with a margin of error at ±3%:
A - 50% B - 40% C - 10%
Now suppose I pick a voter at random from the whole population, not necessarily the sample. What prior probabilities do I assign to that voter choosing A, B, or C?
The reason I'm hung up here is that the prior probabilities given to the individual voter are related to the posterior probability distribution of the entire population after taking the sample - e.g. the population parameter.
If we knew the exact population parameter, we'd know what prior probability distribution to assign to the individual voter - but we don't. Instead, we have a probability distribution of population parameters consistent with the sample, so we have a probability distribution of probability distributions to give to the voter.
The way I'm doing it in my head is to create a massive joint probability distribution - one random variable is the population parameter, the other is the voter breakdown for each hypothetical population. Then, to figure out the overall prior probability I should assign to a random person in the population picking "A", "B", or "C", I just compute the marginal probabilities and call it a day.
However, is this the right approach? These samples are usually done using the frequentist "confidence interval" approach to quantify error, whereas I'm thinking in Bayesian terms which uses "credible intervals" instead. Is there some frequentist/Bayesian subtlety here that can throw me off by mixing them?
Let's suppose you started before the opinion poll with a "flat" prior for the distribution for $a,b,c$ subject to $a+b+c=1$, i.e. a prior density of $\pi_0(a,b,c) = 2 a^0 b^0 (1-a-b)^0$ so $\displaystyle \int_{a=0}^1\int_{b=0}^{1-a} \pi_0 \, db \, da =1$. You could change the initial indices (and constant) but it would make little difference after the opinion poll.
Then taking account of the likelihood of a poll of $1000$ people giving $500$ As, $400$ Bs and $100$ Cs, you would get a posterior density proportional to $$\frac{1002!}{500!400!100!}a^{500} b^{400} (1-a-b)^{100}$$ and a posterior marginal density for $a$ of $\frac{1001!}{500!500!}a^{500} (1-a)^{500}$, for $b$ of $\frac{1001!}{400!600!}b^{400} (1-b)^{600}$, and for $c$ of $\frac{1001!}{100!900!}c^{100} (1-c)^{900}$.
You can then if you wish use these (beta distributed) posterior marginal distributions to construct credible intervals, either by going for the highest density regions or by making the posterior probabilities of being above or below the intervals equal.
In this case, the Bayesian credible intervals will look much the same as the equivalent frequentist confidence intervals, but will have a different interpretation.