Given a number of measurements $\{x_i\}$ with values distributed according to a (known) probability distribution $\rho(x)$ with a theoretical mean $\langle x\rangle = \int dx x\rho(x) = f(y)$ and a calculated experimental mean $\bar{x} = \frac{1}{N}\sum\limits_i x_i$ how does one then find the uncertainty on the experimentally determined value of $y$?
Assume at first that the full probability distribution is known and then, if possible, extend to the case where only the kind of probability distribution is known (restricted to binomial, Poisson, Gaussian & flat).
Naturally one could generate several test distributions, calculate their individual means and from that generate a distribution for $y$ and determine the uncertainty but I would prefer a more theoretical approach (even if I end up with integrals over the distribution that has to evaluated numerically).
Finally, it strikes me that the term uncertainty might not be properly defined (and frankly I do not know if there is a proper definition?) but I was thinking about something along the lines of either the root mean square/standard deviation from the mean or a certain percentage of the measured values that are within the interval [$y-\delta y,y +\delta y$] where $\delta y$ is the uncertainty.
Stated slightly differently: How does one find the distribution for y? (I realize that y has no uncertainty in the theoretical case but if y is determined from samples generated from the probability distribution $\rho(x)$ it will vary from sample to sample thus it must itself have a probability distribution).