Assuming that classifier_1, classifier_2 have an unknown hit ratio $α, β$, what is the probability that $α>β$ if after doing an experiment on a sample of size 20 classifier_1 gets a hit ratio of 80% and classifier_2's ratio is 60%? And what is the probability that $α$ is greater than $β$ by at least 10%? If we have a model in production with a given test accuracy and we subsequently develop a new model that improves those test results, how likely is it worthwhile to change the model?
If I had an estimation of the probability distribution of $α$ and $β$, I could solve the question easily. I know that the probability distribution of $α$ and $β$ is $0$ in 0 and 1. Should I expect it to be centered in $α$ and $β$? How do I obtain those probability distributions?