I have some data to be analyzed. It's histogram looks unimodal, with the support being positive reals between 0 and 100, most of the values huddled up around the mode.I want to be able to quantitatively say that one distribution is a better choice for this data than the other. Say for example, for my given data set, a gamma distribution is better than a beta distribution (with the support proportionally adjusted). Is there any way to do that?
The nature of my problem is such that there is only very limited data, not quite as much as I would have wanted to make a better conclusion. This is why I do not trust the results of the Kernel estimation method. Is there any other method that could be used to estimate the distribution from data?
What you are looking for is called Kolmogorov–Smirnov test. Given a data set and a distribution law, it gives you a statistic which tells you how likely the sample is on the condition that the it is drawn from the distribution: $$P(\text{data }|\text{ distribution})$$ The distribution law with the highest probability wins.