What is the purpose of $\frac{1}{\sigma \sqrt{2 \pi}}$ in $\frac{1}{\sigma \sqrt{2 \pi}}e^{\frac{(-(x - \mu ))^2}{2\sigma ^2}}$?

Question

I have been studying the probability density function...

$$\frac{1}{\sigma \sqrt{2 \pi}}e^{\frac{(-(x - \mu ))^2}{2\sigma ^2}}$$

For now I remove the constant, and using the following proof, I prove that...

$$\int_{-\infty}^{\infty}e^{\frac{-x^2}{2}} = \sqrt{2 \pi }$$

The way I interpret this is that the area under the gaussian distribution is $\sqrt{2 \pi }$. But I am still having a hard time figuring out what the constant is doing. It seems to divide by the area itself and by $\sigma$ as well. Why is this done?

Answer 1

If you consider every possible outcome of some event you should expect the probability of it happening to be $1$, not $\sqrt{2\pi}$ so the constant scales the distribution to conform with the normal convention of ascribing a probability between zero and one.

Answer 2

It is doing that, but observe that you are also stretching in the horizontal direction by the same factor (in the exponential). Say if $\sigma>1$ you are decreasing your area by a factor $\sigma$ but you are increasing it by the same factor because you replace $x$ by $x/\sigma$ (the shift does not change the area of course)

Answer 3

As you have correctly stated, the p.d.f. of the normal distribution is given by $$f(x\mid\mu,\sigma^2)=\frac1{\sigma\sqrt{2\pi}}\exp\left(-\frac12\left(\frac{x-\mu}\sigma\right)^2\right)$$ where the parameter space is $\mathit\Theta=\{(\mu,\sigma^2)\in\Bbb R^2:\sigma^2>0\}$. This is essentially saying that the mean is a value on the real line, and the variance is one on the positive real line.

Now consider the simple case where $\mu=0$ and $\sigma^2=1$. Then the standard normal distribution has p.d.f. $$f(x)=\frac1{\sqrt{2\pi}}\exp\left(-\frac12x^2\right).$$ If we integrate this in the interval $(-\infty,\infty)$, we will get $1$. This is by definition always the case as for all $x\in\mathit X$ (in this instance $\mathit X=\Bbb R$), $$\int_{\mathit X}f(x)\,dx=1.$$ That is, the sum of all the probabilities of $x$ being in each region in $\mathit X$ is $1$. In fact, the constant that makes this happen is so important in statistics (especially Bayesian statistics) that it is given a name: the normalising constant.

A further example is the Beta distribution, with p.d.f. $$f(x\mid\alpha,\beta)=\frac{x^{\alpha-1}(1-x)^{\beta-1}}{\text B(\alpha,\beta)}$$ where $1/\text B(\alpha,\beta)$ is the normalising constant.

Answer 4

The $\exp(-x^2/2)$ part is fundamentally the bell shape of the distribution.

The parameters $\sigma$ controls the width and $\mu$ controls the offset from being centered at $x=0$.

So the whole shape is $$f(x;\mu,\sigma^2) \propto \exp \left( - \frac 1 2 \left( \frac{x-\mu}{\sigma} \right)^2 \right)$$

The term outside the $\exp$ scales the PDF, since the total probability $f(x)$ over all $x \in \mathbb R$ should sum/integrate to 1. This satisfies the probability axiom of unit measure, that is over the domain $x \in \mathbb R$, the probability $X$ takes some real value is 1.