Entropy of a unimodal continuous probability distribution

Question

Among unimodal continuous probability distributions supported on the positive reals and whose mean and mode coincide, which one has the maximal entropy ?

Answer 1

I'll assume that you're referring to the differential entropy of the distribution.

There is no such extremal unimodal distribution.

There are three constraints on the distribution function $f(x)$. Two can be expressed in integral form:

$$\int_0^\infty f(x)=1$$

and

$$\int_0^\infty xf(x)=\mu\;.$$

Ignoring for now the constraint that $f$ is unimodal with mode $x_0$, we get the following Lagrangian:

$$ L[f]=\int_0^\infty\left(f(x)\log f(x)+\alpha f(x)+\beta xf(x)\right)\mathrm dx\;. $$

Varying with respect to $f$ yields

$$ \log f(x)+1+\alpha+\beta x=0\;. $$

Thus, $f$ for this simplified problem would be an exponential function. I'm not sure how to prove this formally, but it seems clear to me that if we add the constraint that $f$ is unimodal with mode $x_0$, the result must be two exponentials decaying away from $x_0$ on either side. But then there's no extremal value of the parameters, because for any given solution you can make the decay slightly slower on both sides while maintaining the mode and the mean, thus slightly increasing the entropy. The limit of this construction is a distribution that's no longer unimodal, but is constant up to the “mode” and then decays exponentially.