Properties of Entropy

162 Views Asked by At

When someone writes $H(X_1, X_2, X_3) = H(X_1) + H(X_2\mid X_1) + H(X_3\mid X_2, X_1)$, how should that last term be interpreted/read? As the joint entropy between 2 variables where variable 1 is $X_3\mid X_2$ and variable 2 is $X_1$? Or As the entropy of $X_3$ conditioned on both $X_2$ and $X_1$? In other words is it:

$$H[ (X_3\mid X_2) , (X_1) ]\text{ or }H[ (X_3) \mid (X_2, X_1) ]$$

Are they the same? If so could someone show me how? If not, could someone tell me the correct way to read/interpret/write both possible interpretations?

1

There are 1 best solutions below

2
On BEST ANSWER

The intended interpretation in this formula is $H(X_3\mid (X_2,X_1))$. Your other option won't work because $X_3\mid X_2$ is not a random variable.