Gaussian distribution determined by first two moments

Question

When said that Gaussian distribution is determined by it's mean and variance. How is that different of other distributions? Almost every distribution which I can think of has this property. For example if we know the mean of exponential, Poisson distribution then we know the whole distributions.

Answer 1

I agree with the comment by @eigenchris for 'well-known' distributions encountered early on in a probability course. However, one does not have to venture too far into the study of probability distributions to find examples in which knowing the population mean and variance does not easily specify the distribution.

It is useful to make a distinction between the parameters of the distribution of a random variable $X$ and other quantities such as $\mu =E(X), \sigma^2=V(X),$ and $E(X^2),$ (sometimes called 'moments') which in some sense might be said to 'determine' the distribution. Here are a few examples:

UNIFORM: If $X \sim Unif(\alpha_1, \alpha_2)$, then the endpoints $\alpha_1$ and $\alpha_2$ of the support interval are usually taken as the parameters of the distribution. However, if specified, the mean $\mu = E(X) = (\alpha_1 + \alpha_2)/2$ and variance $\sigma^2 = V(X) = (\alpha_2 - \alpha_1)^2/12$ could be used to find the parameters $\alpha_1$ and $\alpha_2$ in terms of $\mu$ and $\sigma^2.$

GAMMA: If $X \sim Gamma(\alpha, \theta$), then $\alpha$ is the shape parameter and $\theta$ is the scale parameter. Again, if $\mu = \alpha\theta$ and $\sigma^2 = \alpha\theta^2$ are known, then we could easily solve to find the parameters $\alpha$ and $\theta$ in terms of $\mu$ and $\sigma^2.$

In both uniform and gamma distributions, it is usually more natural or intuitive to think in terms of the parameters, even though they are straightforwardly determined by $\mu$ and $\sigma^2.$

In some other families of distributions, the relationship between moments $\mu$ and $\sigma^2$ and the more natural parameters is not expressed so transparently.

BETA. This family of distributions has two parameters $\alpha$ and $\beta.$ These distributions have support $(0, 1)$. Very roughly speaking $\alpha$ controls the 'shape' of the distribution near $0$ and $\beta$ controls shape near $1$. Here $\mu = \alpha/(\alpha + \beta)$ and $\sigma^2 = \frac{\alpha\beta}{(\alpha + \beta)^2(\alpha + \beta + 1)}.$ It is possible, but not really easy or intuitive to use the moments to determine the parameters.

WEIBULL. This family of distributions has a shape parameter $\kappa$ and a scale parameter $\lambda.$ It is often used in reliability theory and economics. Here $\mu = \lambda \Gamma(1 + 1/\kappa),$ where $\Gamma$ is the gamma function; $\sigma^2$ is expressed in terms of a somewhat more complex formula involving two $\Gamma$ functions and the parameters (see Wikipedia).

In applications of these last two families of distributions, it is much more natural to think in terms of the parameters than in terms of the moments, even though numerical methods can be used to find the parameters if specific values of the moments are given.