What is the distribution that represents our knowledge of a normal distribution, from a given sample?

Question

Suppose we have a sample $\{x_1,...,x_n\}$ of points drawn from a gaussian with completely unknown mean and variance. Now, we draw an additional point from that distribution, without looking at it. What is the probability distribution of that additional point, given our knowledge?

Note that simply taking the gaussian defined by the maximum likelihood estimators doesn't cut it. The latter is merely the most likely source distribution. The distribution that best represents our knowledge has to be a weighted sum of all possible gaussians, each weighted by its likelihood.

Answer 1

After some computation, the answer seems to be

$\dfrac{1}{\sqrt{(n+1) \sigma ^2} B\left(\dfrac{1}{2},\dfrac{n-2}{2}\right)}\left(\dfrac{(x-\mu )^2}{(n+1) \sigma ^2}+1\right)^{\dfrac{1-n}{2}}$

, where

$\mu = \dfrac{1}{n} \sum _{i=1}^n x_i\quad , \quad \sigma ^2 = \dfrac{1}{n}\sum _{i=1}^n x_i^2-\mu ^2$.

This actually does look a lot like Student's t-distribution! Note that it is a proper distribution only when $n > 2$.

Answer 2

Student distribution might be what you are looking for, it takes into account that we do not know the variance of the Gaussian distribution a-priori resulting in a distribution wider than the Normal distribution.