Prove 0<=entropy<=1

Question

The entropy of a Bernoulli random variable X with $P(X=1)=q$ is given by

B(q)=-qlog(q)-(1-q)log(1-q)

How do we prove 0<=B(q)<=1?

Answer 1

To show $B(q) \ge 0$ we need to show that $q\log(q)+(1-q)\log(1-q) \le 0$. This is true as both $q,1-q \ge 0$ and $\log(q),\log(1-q) \le 0$.

To show $B(q) \le 1$, we need to show $-q\log(q)-(1-q)\log(1-q) \le 1$. To do this we find the minimum of $q\log(q)+(1-q)\log(1-q)$. If we can show this is greater than or equal to $-1$ we are done. Taking the derivative with respect to $q$ gives $1+\log(q)-1-\log(1-q) = \log(q)-\log(1-q) = \log(\frac{q}{1-q})$. This equals $0$ when $\frac{q}{1-q} = 1$ so $q=\frac{1}{2}$. The second derivative test will show this is the minimizer, so the minimum is $\log(\frac{1}{2})=-\log(2) > -1$.