How does the presenter in this video derive this formula?

79 Views Asked by At

I am watching this coursera video on entropy (in the information theory sense of the word). Right around the two minute mark the presenter shows two forms for H(p). The first (after the equals sign) is a definition. But I do not understand how he derives the second form (after the second equals sign). I guess I am not that familiar with the bracket notation [] that he uses to define the "expectation over the surprise." What does it mean for X to be "over" Y? What does that have to do with this notation?

enter image description here

1

There are 1 best solutions below

0
On BEST ANSWER

If you are asking about the equality $$E_p\left[\log_2 \frac1{p_x}\right] = -\sum_x p_x\log_2 p_x$$ then it is just the definition of expected value and using $\log_2\frac1x=-\log_2x$. (Which, of course, works for logarithms at any base, not just base two.)