Differential entropy measure that is sensitive to the amount of dimensions?

53 Views Asked by At

In information theory, if we have $n$ discrete bits, i.e. variables with a domain of two elements, and we have a uniform distribution over each element, then the entropy of the total system equals $n$. This is a discrete notion of entropy.

There is also a notion of differential entropy. Suppose we have $n$ iid uniformly distributed variables on $[0,1]$. Then we can't talk about the discrete entropy of the system, but we can talk about their differential entropy (which formulaically is the same).

However, in that example, the differential entropy assigns entropy $0$ to this uniform system, and it doesn't depend on the number of dimensions. But intuitively, we have more uncertainty in some sense, if we add more variables.

Is there a notion of differential entropy that assigns entropy $n$ to a system of $n$ iid uniform variables on the unit interval?