volume of region in a probability simplex with shannon entropy > c?

212 Views Asked by At

Let $S_n = \{(p_1, \cdots, p_n)| p_i\geq 0, \sum_{i=1}^n p_i = 1\}$. I.e. $S_n$ is a $n-1$-simplex. Let $$ E(p_1, \cdots, p_n) = -\sum_{i=1}^n p_i\ln p_i $$ be the Shannon entropy.

I am stuck with the question, what is the $n-1$-Volume of $\{(p_1, \cdots, p_n)\in S_n \mid E(p_1, \cdots, p_n)>a\}$?

I mean I am interest in integral $$ \int_{X}dV $$ where $X= \{(p_1, \cdots, p_{n-1}) \mid p_i\geq 0, p_n = 1-\sum_{i=1}^{n-1}p_i\geq 0, E(p_1, \cdots, p_n)>a\}\subset {\mathbb R}^{n-1}$

Any non-trivial bounds are also interesting.

1

There are 1 best solutions below

1
On

Just some values from numerical simulation. The horizontal axis is the entropy (in nats), and the vertical axis is the normalized volume.

enter image description here

The entropy tends to get concentrated around the mean, which is given by

$$ e(n)=\psi_0(n+1) + \gamma -1 \approx \log n -0.423 +\frac{1}{2n} $$

where $\psi_0$ is the digamma function.