Suppose i have N types of flowers and I want to find which flower people find the most beautiful. Each day, I have a new judge come in to rate (from 0 to 1) a flower of my choice. I have M>>N days to decide estimate which flower is the most beautiful.
If I let each flower be judged the same number of times, then in the end, the MLE estimate for the most beautiful flower would be the flower with the highest average rating.
However, are there any superior algorithms to determining which flower to get the judges to rate if we're doing this in sequence? What if we changed the question to be: which flowers have beautifulness > 0.8?
I feel like there should be a way but Im also unsure how. Distribution wise, let us assume the rating of the flower follows a beta distribution.
EDIT: I define a superior algorithm as one that, under the above assumptions, estimates the most beautiful flower, with lowest cross entropy loss. Or replace cross entropy with something similar if it leads to a mathematically simpler solution.