Is this some entropy I haven't heard of?

145 Views Asked by At

For a discrete finite probability distribution $p(s)$ the function $$\sum\limits_s p(s)\log ^2 p(s)$$ looks like the Shannon entropy but has a square on the $\log$. Is there a name for this? Or it is meaningless?

1

There are 1 best solutions below

0
On

This relative of Shannon's entropy fails to display additivity and is not maximized for a uniform distribution. These properties are often desirable in a measure of information and are properties of Shannon's entropy.

For instance:

Additivity: $H(.25,.25,.25,.25) \neq H(.5,.5) + H(.5,.5)$.

Uniform Maximum: $H(.5,.5)\approx .48 \space \text{while} \space H(.9,.1)\approx .54$

In fact, here is the plot of the function for two outcome R.V. Uniform is a local minimum.

enter image description here