Binary Entropy solving over uniform distributions

99 Views Asked by At

Compute the Binary Entropy for X is uniform over the set {1, 2, 3, . . . , 80}

$$H(X) = -\sum_{i=0}^n p(x)log _2p(x) $$

I am pretty confused on this topic if someone could explain an easy way to solve this.

I know their probabilities are 1/n so in this case it will be 1/80.

1

There are 1 best solutions below

2
On BEST ANSWER

$$ H(X) = - \sum_{i=1}^{80} \frac{1}{80} \log_2\frac{1}{80} = - 80 \frac{1}{80} \log_2\frac{1}{80} = - \log_2 \frac{1}{80} = \log_2 80 $$

Note that this actually maximises the entropy, the uniform distribution has this property.