Is there intuition for "mathematical" entropy. I know that physical entropy tracks the order in a dynamical system, for thermodynamics. As entropy goes up, general randomness and disorder goes up. However, what is entropy in the mathematical sense? There seem to be many kinds of entropy, but what is there a general defining characteristic or use that is common among them all? I'd appreciate some intuition for this interesting subject. (Edit: Alex R. provided a great link for the above)
Pertaining to my interests, what is Entropy over a Fractal? Is this Entropy taken by treating the Fractal as a probability distribution? What purpose does taking the Entropy of a Fractal serve?