Integrate a general normal pdf

51 Views Asked by At

I have basic 'college calculus' and am reading through 'Statistical Rethinking,' a very illuminating text on Bayesian stats in R. A general Gaussian pdf is given on p274:

$$p(x) = (2πσ^2)^{-1/2} \exp(-(x-μ)^2/(2σ^2))$$

and the entropy of the Gaussian is $H(p) = -∫p(x)\log (p(x))dx = (1/2)\log(2πσ^2)$

....with no further workings. I've tried online integral calculators but they either give up or (when I try searching for hints by inputting $∫p(x)dx$ or $\int logp(x)dx$ separately) they come back with stuff like the Gauss error function which is outside my comfort zone. I do get a few clues such as taking out the constant $(2πσ^2)^{-1/2}$, or substituting for $(x-μ)/(\sqrt2σ)$, but how to boil it all down to $\frac12 \log(2πσ^2)$ is beyond me.

Is anyone out there patient enough to help me join the dots?

[screenshot of text]

1

There are 1 best solutions below

0
On BEST ANSWER

The good news is that you don't need to calculate the individual integrals to get to the final result

$p(x)$

Just for context, some results you may know about

$$ p(x) = \frac{1}{\sqrt{2\pi \sigma^2}}e^{-(x - \mu)^2/2\sigma^2} \tag{1} $$

which is normalized

$$ \int_{-\infty}^{+\infty} {\rm d}x~p(x) = 1 \tag{2} $$

also,

\begin{eqnarray} \int_{-\infty}^{+\infty} {\rm d}x ~\frac{(x - \mu)^2}{2\sigma^2} p(x) &=& \mathbb{E}\left(\frac{(X - \mu)^2}{2\sigma^2}\right) \\ &=& \frac{1}{2\sigma^2}\mathbb{E}\left((X - \mathbb{E}(X))^2\right) \\ &=& \frac{1}{2\sigma^2} \mathbb{V}{\rm ar}(X) = \frac{\sigma^2}{2\sigma^2} = \frac{1}{2} \tag{3} \end{eqnarray}

$H$

Now to the entropy. Note that

\begin{eqnarray} H &=& -\int_{-\infty}^{+\infty} {\rm d}x~p(x)\ln p(x) \\ &\stackrel{(1)}{=}& -\int_{-\infty}^{+\infty}{\rm d}x~ p(x) \ln \left[ \frac{1}{\sqrt{2\pi \sigma^2}}e^{-(x - \mu)^2/2\sigma^2}\right] \\ &=& -\int_{-\infty}^{+\infty}{\rm d}x~ p(x) \left[ - \ln\left(\sqrt{2\pi \sigma^2}\right) - \frac{(x - \mu)^2}{2\sigma^2}\right] \\ &=& \ln\left(\sqrt{2\pi \sigma^2}\right)\int_{-\infty}^{+\infty}{\rm d}x~ p(x) + \int_{-\infty}^{+\infty}{\rm d}x~ \frac{(x - \mu)^2}{2\sigma^2} p(x) \\ &\stackrel{(2),(3)}{=}& \ln\left(2\pi \sigma^2\right)^{1/2} + \frac{1}{2} \\ &=& \frac{1}{2} \left[\ln(2\pi\sigma^2) + \ln e\right] \\ &=& \frac{1}{2} \ln(2\pi e\sigma^2) \end{eqnarray}