Entropy: Is $H(X_{1},X_{2}) = H(X_{1})$ true?

79 Views Asked by At

Question: If $X_{1}, X_{2}$ are two discrete random variables. $X_{1}, X_{2}$ have the same probability distribution can we then deduce that:

$H(X_{1}, X_{2}) = H(X_{1})$

is true?

Remark: $H(X)$ denotes the entropy of the random variable $X$.

1

There are 1 best solutions below

1
On BEST ANSWER

NO, we can not.

$H(X_1,X_2)=H(X_1)+H(X_2|X_1)$, but if $X_1,X_2$ are independent and $H(X_1)>0$ then

$H(X_2|X_1)=H(X_2)=H(X_1)>0$

Hence $H(X_1,X_2)=H(X_1)+H(X_2)=2H(X_1)>H(X_1)$.

Therefore $H(X_1,X_2)=H(X_1)$ is not valid.