Finding minimum of expression over all codings

23 Views Asked by At

If a binary prefix code has input distribution $p_x$, with length $n_x$ for each code, how can I show that the minimum of $D=\sum_xp_x2^{n_x}$ is bounded by:

$$\left(\sum_x\sqrt{p_x}\right)^2\le D\le2\left(\sum_x\sqrt{p_x}\right)^2$$

This seems related to the noiseless coding theorem, with both sides raised to the power $2$. However, I can't see how to get the terms on the left and right.

1

There are 1 best solutions below

0
On

I think I've figured it out. The Shannon Coding gives:

$$-\log _{x}p_{i}\leq n_{x}<-\log _{x}p_{x}+1$$

Throwing that into the formula for $D$ led to the desired expression.