I flip a fair coin 4 times, and store the outcomes in a random variable X:
X = (X0, X1, X2, X3)
Then I have: Y = (X0, X1)
and
X' = X3 * 8 + X2 * 4 + X1 * 2 + X0. Let Z = X' mod 6.
I have to compute the entropy H(Y,Z). I already computed the entropies H(X), H(Y) and H(Z) with the known entropy formula (the negative of the sum of the probability times the logarithm of the probability), but I have no idea how to compute a "joint entropy".
What is the way to go?
The idea is to combine all possible outcomes of Y (00, 01, 10, 11) and Z (0, 1, 2, 3, 4, 5) and put them in a table. Then, you compute all joint probabilities ( P((Y=00) AND (Z=0)), and so on. Then, you simply apply the formule of entropy.
The final is result is 3.5.