I have a problem finding the probability density function of entropy of a normally distributed sample.
It is known that the entropy of a gaussian variable $X$ equals $H=h(X)={1\over2}\log(2\pi es^2)$, where $s^2$ is the variance.
Now, the probability density function of sample variance $s^2$ of a sample of size $N$, taken from a normally distributed population with variance $\sigma^2$ is known to be
$$ f(s^2) = {({N\over{2\sigma^2}})^{N-1\over{2}}\over{\Gamma({N-1\over{2}}})}e^{-Ns^2\over{2\sigma^2}}(s^2)^{N-3\over2}\quad . $$
So, using the transformation technique it should be possible to obtain the probability density function of entropy $H$: $$ g(H)= \big|{d\over dH}h^{-1}(H)\big|*f(h^{-1}(H))\quad. $$ In our case we have ${h^{-1}(H)}={e^{2H-1}\over{2\pi}}$ and the derivative ${{d\over dy}h^{-1}(H)}={e^{2H-1}\over{\pi}}$.
If we put it all toghether we obtain the probability density function
$$ g(H)=\frac{2\cdot N^{(N-1)/2}}{\Gamma({N-1\over{2}})}\cdot\exp\left(-N\frac{e^{2H-1}}{4\pi\sigma^2}\right)\cdot \left(\frac{e^{2H-1}}{4\pi\sigma^2}\right)^{(N-1)/2}. $$
The problem, which arises here is the fact that the function $g(H)$ when simulated in Matlab for varius meaningful values of H (~10) and N (~100) gives bogus results (usually 0 or NaN). This is obviously because of the very ugly exponential term in the equation, but it is still strange to me that I can't get correct results using this method. Any ideas on what did I do wrong or how can this be solved?
The trouble might come from the fact that, for $N=100$ and $\sigma^2=1$, the mode of the density $g$ is around $H=1.75$ and that you are looking at $g(H)$ for $H=10$, where $g(H)$ is about $10^{-K}$ times its value at the mode, with $K\approx1.7\cdot10^9$...
Plots of $g$ indicate that most of its mass is concentrated between $1.6$ and $1.9$ hence, for practical purposes, values of $H$ out of, say, the interval $(1.5,2.0)$ are never seen. Here is a plot of (some analogue of) $g$ (non normalized) around its mode:
plot exp(200x-3exp(2x)) from 1.5 to 2$\qquad\qquad\qquad\qquad\qquad$