Why are these two expressions involving probabilities equal?

Question

I have encountered an equation I do not understand:

\begin{multline*} -\sum_{x}\sum_{y}p(x,y)(\log_{2}p(x)+\log_{2}p(y)-\log_{2}p(x,y))\\ =-\sum_{y}p(y)\log_{2}p(y)+\sum_{x}p(x)\sum_{y}p(y|x)\log_{2}p(y|x) \end{multline*}

Mainly, I do not understand how we went from ordinary probabilities to conditional ones.

The post Mutual Information: How these two equations are equal? is similar, but conditional probability is not used.

Answer 1

The middle term on the left side is the first term on the right side. (Just note that $\sum_x p(x,y)=p(y)$). In the other two terms on the left just put $p(x,y)=p(y|x)p(x)$ to complete the argument.

Answer 2

Assuming the domain of $p$ is finite so that we can ignore any issues relating to switching the orders of summation, \begin{multline*} \sum_{x}\sum_{y}p(x,y)\left(\lg p(x)+\log p(y)-\lg p(x,y)\right)\\ =\sum_{x}\lg p(x)\sum_{y}p(x,y)+\sum_{y}\lg p(y)\sum_{x}p(x,y)-\sum_{x}\sum_{y}p(x,y)\lg p(x,y)\\ =\sum_{x}p(x)\lg p(x)+\sum_{y}p(y)\lg p(y)-\sum_{x}\sum_{y}p(x)p(y\mid x)\lg(p(x)p(y\mid x))\\ =\sum_{x}p(x)\lg p(x)+\sum_{y}p(y)\lg p(y)-\sum_{x}p(x)\sum_{y}p(y\mid x)\left(\lg p(x)+\lg p(y\mid x)\right)\\ =\sum_{x}p(x)\lg p(x)+\sum_{y}p(y)\lg p(y)-\sum_{x}p(x)\lg p(x)-\sum_{x}p(x)\sum_{y}p(y\mid x)\lg p(y\mid x)\\ =\sum_{y}p(y)\lg p(y)-\sum_{x}p(x)\sum_{y}p(y\mid x)\lg p(y\mid x). \end{multline*}