information entropy - why does a skewed probability distribution lead to smaller entropy?

Question

System A: n=0.25prob. e=0.25. w=0.25 s=0.25 System B: n=7/16; e= 1/4; w=1/4; s=1/16;

using information entropy formula system B has a smaller entropy. I thought that a distribution of probabilities meant that there would be more disorder.

Or is it because more of the fact that the order lies in the values of the probabilities not in the diversity of probabilities- in that if you make one state so much more likely than all others, then it basically guarantees that it happens.

Answer 1

the order lies in the values of the probabilities not in the diversity of probabilities

Yes, the "diversity of probabilities" does not matter. A uniform distribution (equiprobable random variables) have maximum entropy (important property), because you have maximum uncertainty. The system $B$ has one element with high probability and one with low probability, its average information content is less than that of system $A$.