Proof $ I(X;Y) = H(X) - H(X|Y) $

50 Views Asked by At

$I(X;Y) = H(X) - H(X|Y)$

proof

$ I(X;Y)=\sum_{x,y}p(x,y) \log \frac{p(x,y)}{p(x)p(y)} $

$=\sum_{x,y}p(x,y) \log \frac{p(x|y)}{p(x)}$

$= \sum_{x,y}p(x,y) \log p(x|y) + \sum_{x,y}p(x,y) \log p(x) $

$= - H(X|Y) + \sum_{y} p(y|x) \sum_x p(x) \log p(x) $

Can you please help me get the term H(X)

1

There are 1 best solutions below

0
On BEST ANSWER

$$-\sum_{x,y} p(x,y) \log p(x) = - \sum_x \log p(x) \sum_y p(x,y) = - \sum_x (\log p(x)) p(x) = H(X).$$