An upper bound of $-p \log p$ with $p > 0$ very small

Question

I am looking for an upper bound of $-p \log p$ in terms of probability $p$ with $1 > p>0$ when $p$ is very close to $0$ (at least $p < 1/e$).

It would be really great if I have a tight one.

I know a good upper bound when $p$ is close to 1; the upper bound is $1-p$.

Answer 1

We know that for any $p>0$ we have$-p\log p \le \frac{1}{e}$. Let $\alpha\in{\mathbb R}$. The inequality holds with $p^{1-\alpha}$ instead of $p$ hence we have \begin{equation}-(1-\alpha)p^{1-\alpha}\log p = - p^{1-\alpha}\log(p^{1-\alpha})\le \frac{1}{e}\end{equation} When $\alpha < 1$ we deduce \begin{equation}-p\log p\le \frac{p^\alpha}{e(1-\alpha)}\end{equation} For example when $\alpha = 1/2$, \begin{equation}-p\log p\le \frac{2\sqrt{p}}{e}\le\frac{3}{4}\sqrt{p}\end{equation}

Answer 2

We use the well known inequality $logp\geq\,1-\dfrac{1}{p}$. Then we multiply by $-p$ and we get $-plogp\,\leq\,-p+1$. So your upper bound holds for all $p$ not just for $p$ close to zero.

Now using L'Hospital rule we calculate $\displaystyle \lim_{p \to 0}-\dfrac{logp}{1/p}=\lim_{p \to 0}p=0.$,

So we can get an upper bound as small as we want by taking $p$ sufficiently close to zero.

Answer 3

You can have good upper bounds building $P_n(p)$, the $[n,n]$ Padé approximant of $-p \log(p)$ built around $p=\frac 1e$. You need $n \geq 2$.

For example $$P_2(p)=\frac{1}{e}\times\frac {1+\frac{e}{3} \left(p-\frac{1}{e}\right)-\frac{5e^2}{9} \left(p-\frac{1}{e}\right)^2 } {1+\frac{e}{3} \left(p-\frac{1}{e}\right)-\frac{e^2}{18} \left(p-\frac{1}{e}\right)^2 } > -p\log(p) \quad \text{for} \quad p \in \left(0,\frac 1e\right)$$ To give you an idea $$\Phi_2=\int_0^{\frac 1e} \Big[P_2(p)+p\log(p)\Big]^2\,dp=5.77\times 10^{-5}$$ $$\Phi_3=\int_0^{\frac 1e} \Big[P_3(p)+p\log(p)\Big]^2\,dp=6.58\times 10^{-6}$$

Any $P_n$ will be larger than $-p \log(p)$ in the range.

$e\, P_n(0)$ generates the sequence $$\left\{\frac{2}{11},\frac{8}{93},\frac{19}{374},\frac{107}{3180},\frac{39}{1628},\frac{293}{16338},\cdots\right\}$$ which are closer and closer to $0^+$