I have the following exam question for a multimedia exam in college:
Assume that you roll a single ordinary six-sided die twice, and observe that the second number rolled is greater than the first. In an information theoretic sense, what is the information content associated with this outcome?
If the information content associated with an event is -log2Pe (- log to base 2 times probability of event e), would the following be true:
First Roll: -log2( 1 / 5 ) //First number rolled can be any number except 6
Second Roll: -log2( 1 / 5 ) //Second number can be any number but the first
Therefore the information content of this outcome =
-(log(1/5)/log(2)) + -(log(1/5)/log(2)) = 4.64385618977
or would it be
x = probability of second roll given the first
-log(x / log(2))
If that is the case, how do I work out the value of x.
The probability that the second roll ist greater than the first is $\frac{15}{36}=\frac{5}{12}$. You can calculate the information content as $-\log_2 p$ then.