Let's say there's a simple roulette in play. It has no colors, just numbers 1-36(i = 36).
a) Find the entropy of the roulette, if we're interested in where the ball stopped
b) Find the entropy of the roulette, if we're interested in if we won or not.
I'm having big trouble understanding how to adjust entropy calculations based on what we're interested in.
For a) I just plugged in P(X) = 1/36 into this equation:
I got 5.169 bits.
Problem is I cant wrap my mind around how to adjust the entropy equation for b)
The chances of winning (P(x) = 1/36) and the chances of losing are P(y) = 35/36).
But calculating two entropies, one for winning and one for losing is just calculating the same thing I did in a).
From what I understand, entropy tells me the randomness of a system. If I have 36 numbers, each have the same probability of "happening". I cant wrap my head around how we adjust calculations/computations if we're "interested" in something. Like how does an entropy even change if we're interested in winning or interested in which number "happened".
I wish I could provide further explanation for the question but that's the whole context. Is this an assignment problem? Should I request clarification on what "interested" means? Or am I just lost.
Entropy of a random variable is what is shown in the equation you linked. So both questions are asking you to calculate entropy of some random variable. What is the random variable is the question you should ask when you are told to calculate entropy.
In the first case, the random variable is the number that shows up on the roulette. Assuming the uniform distribution, the random variable is $X \sim Unif(\{1, 2, \ldots, 36\})$. Thus, as you rightly calculate, the answer is $\log_2{36} = 5.17$.
The second question, I am assuming that they are interested in the random variable $Y$ that denotes whether you won or lost. So the probability of winning is $\frac{1}{36}$ and losing is $\frac{35}{36}$. So the random variable $Y$ is a Bernoulli random variable. We get
$$H[Y] = -\left(\frac{1}{36}\log_2{\frac{1}{36}} + \frac{35}{36}\log_2{\frac{35}{36}}\right) = ...$$