Checking efficiency of randomness with entropy

Question

I was going through random numbers and found that the randomness of certain observations is measured by the entropy as given in here. Here, $p(x_i)$ is the probability that $x_i$ will take place. But if I have fair dice then $p(x_i)$ is $\frac{1}{6}$. So, I am assuming that less is the entropy, better is the randomness. Is my conclusion correct? Also, entropy is calculated in terms of - so while determining the efficiency should I consider this negative sign also or only the positive number? I hope my question is clear. Any help will be appreciated!

Answer 1

Your conclusion is wrong. Entropy in a measure of randomness it is correct but it increases if the randomness increases. There are many different characterizations to maximize the entropy. For example from all densities with equal variance Gaussian density is the one which maximizes the entropy.

On the other hand given a number of events, uniform distribution maximizes the entropy. This means if you have a fair dice it has the maximum entropy and also maximum randomness. If some events are correlated to the others, then entropy decreases as well as the randomness because you can estimate it using the correlation patterns! Read: http://en.wikipedia.org/wiki/Maximum_entropy_probability_distribution