Decomposing fractions

54 Views Asked by At

I am not sure how these two terms are equal from this wiki:

$$ I(X,Y) = KL(p(x,y) || p(x)p(y)) = \sum_{x,y}p(x,y) \log[\frac{p(x,y)}{p(x)q(y)}] \\ I(X,Y) = \sum_{x,y}p(x,y) \log[\frac{p(x,y)}{p(y)}] - \sum_{x,y}p(x,y) \log(p(x)) $$

In other words, I can rewrite the second line as (ignoring the summation and $p(x,y)$ for now) $$[\log(\frac{p(x,y)}{p(y)}) - \log(p(x))] =[\log(\frac{p(x,y)}{p(y)}) + \log(1/p(x))] $$

From here, just following the rules of fraction addition, I'd get: $$ = [\log(\frac{p(x,y)p(x)}{p(y)p(x)}) + \log(\frac{p(y)}{p(x)p(y)})] \rightarrow [\log(\frac{p(x,y)p(x) + p(y)}{p(y)p(x)})]$$

How does $[\log(\frac{p(x,y)p(x) + p(y)}{p(y)p(x)})]$ equal $\log[\frac{p(x,y)}{p(x)q(y)}]$

What am I doing wrong?

1

There are 1 best solutions below

0
On BEST ANSWER

Note that if $A>0$ and $B>0$, then we have $$\log A + \log B = \log (A\cdot B)$$

Note that:

$$\log A + \log B \ne \log (A+B)$$

in general.

$$\log \left( \frac{p(x,y)}{p(y)} \right)+\log \frac1{p(x)}=\log \left( \frac{p(x,y)}{p(y)p(x)} \right)$$

where I have multiplied the input of the logarithms.