Question: If $X_{1}, X_{2}$ are two discrete random variables. $X_{1}, X_{2}$ have the same probability distribution can we then deduce that:
$H(X_{1}, X_{2}) = H(X_{1})$
is true?
Remark: $H(X)$ denotes the entropy of the random variable $X$.
Question: If $X_{1}, X_{2}$ are two discrete random variables. $X_{1}, X_{2}$ have the same probability distribution can we then deduce that:
$H(X_{1}, X_{2}) = H(X_{1})$
is true?
Remark: $H(X)$ denotes the entropy of the random variable $X$.
NO, we can not.
$H(X_1,X_2)=H(X_1)+H(X_2|X_1)$, but if $X_1,X_2$ are independent and $H(X_1)>0$ then
$H(X_2|X_1)=H(X_2)=H(X_1)>0$
Hence $H(X_1,X_2)=H(X_1)+H(X_2)=2H(X_1)>H(X_1)$.
Therefore $H(X_1,X_2)=H(X_1)$ is not valid.