I have a basic question about the Shannon Entropy formula. In fact it's so dumb that I didn't dare ask it in the class because I don't understand the text books.
Here's the formula:
$$H(X) = -\displaystyle\sum_{x} p(x)\log p(x).$$
And here's my question: what function is p(x)? Is it frequency, and if so, frequency of what?
It is a probability mass function. In particular, $p(x_0)$ is the probability of event $x_0$ occurring.