Entropy measures the "surprise" one experiences when uncovering a the actual value of a random variable as $$-\sum_i p_i \log_2 p_i$$
E.g., if we observe Red 8 times, Green 4 times, and Blue 4 times, and estimate the underlying probabilities from those observations, we say that the variable has entropy $$H(X)=\frac{1}{2}\times 1 + \frac{1}{4}\times 2 + \frac{1}{4}\times 2 = \frac{3}{2}$$
However, suppose that the information about X is supplied by several sources. How do I apportion these 1.5 bits of information among them?
E.g., suppose the source Tulip supplied Red 4 times and Blue 4 times, source Daisy supplied Red 4 times and source Iris supplied Green 4 times.
What is the value of each source to me?
Clearly Tulip is the most valuable (it produces half of the observations and all Blues), Iris is the second (it produces all the Greens), and Daisy is the third.
EDIT:
A more formal setting is:
There is a random variable $X$ taking values from set $C$. It is observed together with "source" $S$ taking values from a smaller set $F$. I want to express the entropy of $X$ as a sum $\sum_s V(s)P(S=s)$ where $V$ is the "value" of the source $s$. The obvious condition on $V$ are: $V(s_1)>V(s_2)$ if $$H(X|S=s_1)P(S=s_1)>H(X|S=s_2)P(S=s_2)$$ or $$H(X|S\ne s_1)P(S\ne s_1)<H(X|S\ne s_2)P(S\ne s_2)$$