A simple inequality involving the entropy function $H(x)$.

119 Views Asked by At

I came across the following simple inequality when I was working on a proof. I would like to know if it's a known inequality and used anywhere.

$H(x) > (\ln x)\times (\ln (1-x)), 0 < x < 1.$

The proof goes as follows:

\begin{eqnarray} H(x)&=&-x\ln x -(1-x)\times \ln(1-x)\\ &=& -x \ln x + (x-1)\times \ln (1-x), \end{eqnarray} and since $-x \ln x > 0, 0 < x < 1,$ and $x-1 \ge \ln x, \forall x >0,$ \begin{eqnarray} H(x)&\ge& (x-1)\times \ln (1-x)\\ &\ge &(\ln x)\times (\ln (1-x)). \end{eqnarray}

Any comments would be welcome.

1

There are 1 best solutions below

0
On

It turns out that this inequality was proved in the following article:

M Bahramgiri and O Naghshineh Arjomand. A simple proof of the entropy inequality. RGMIA research report collection, 3(4), 2000.

The same article also provides an upper bound.

Here is the 2-sided inequality that is proved in this article.

$(\ln x)(\ln (1-x)) \le H(x) \le (\ln x)(\ln (1-x))/\ln 2, 0 < x < 1.$

A more elementary proof of the left inequality is provided as follows:

Writing out the entropy function and dividing both sides of the desired inequality by $(\ln x)(\ln (1-x)),$ (which is positive), gives

$1 \le \frac{-x}{\ln (1-x)} + \frac{-(1-x)}{\ln x}, 0 < x < 1.\,\,\, (1)$

Using the logarithmic inequality $\ln y > (y-1)/\sqrt{y};\, 0 < y < 1$ with $y=x$ and $y= 1-x$ implies

$\sqrt{x} +\sqrt{1-x} < \frac{-x}{\ln (1-x)} + \frac{-(1-x)}{\ln x}\,\,\, (2)$.

Now, it is not difficult to verify that $1\le \sqrt{x} +\sqrt{1-x}, 0 < x < 1$ and this together with $(2)$ implies $(1),$ which then implies the desired entropy inequality.