Suppose $t$ is a non empty set divided in $j$ different classes and consider $p_j$ the relative frequency of the elements of class $j$ with respect to all the elements of the set $t$. Consider the entroyp formula $\sum_j p_j\times \log p_j$.
I would like to prove that
$-\log j\leq $ $\sum_j p_j\times \log p_j$ $\leq 0 $
$\sum_j p_j\times \log p_j$ $\leq 0$
should follow from the fact that for every product $ p_j\times \log p_j $ in the summation, it is at most 0 (when $p_j=1$ or $p_j=0$, and less then 0 if it has a value in the interval (0,1) ). I do not know how to show that
- $-\log j\leq $ $\sum_j p_j\times \log p_j$
and
- $-\log j = $ $\sum_j p_j\times \log p_j$ if and only if $p_j=1/j$ (implying that the minimum value of the summation is reached when every relative frequency is exactly the same.