proof of relationship between $H(S)$ and $H(S^n)$

66 Views Asked by At

in order to proof relationship between $H(S)$ and $H(S^n)$ in this video lecture Professor writes an equality:

$$-\sum_{i\in S^n} p(\sigma_i)\log_2 p_{i_1} = -\sum_{i\in S^n} {p_{i_1}}{p_{i_2}}...{p_{i_n}}\log_2 p_{i_1}$$

where $p(\sigma_i) = {p_{i_1}}{p_{i_2}}...{p_{i_n}}$. and then he concluded that(in mimnute 26:00):

$$-\sum_{i=0}^{K-1} {p_{i_1}}{p_{i_2}}...{p_{i_n}}\log_2 {p_{i_1}} = $$ $$-\sum_{i_1=0}^{K-1}{p_{i_1}}\log_2 {p_{i_1}}\sum_{i_2=0}^{K-1} {p_{i_2}} \sum_{i_3=0}^{K-1} {p_{i_3}} ... \sum_{i_n=0}^{K-1} {p_{i_n}} $$

and i can't figure out this last equality! I appreciate if anyone can provide a proof for this last equation.

1

There are 1 best solutions below

0
On BEST ANSWER

The sum in the left side of your equality should rather be $-\sum_{i\in S^n}$, that is, run over all values on all the indexes: $$-\sum_{i\in S^n} (\cdot)= -\sum_{i_1}\sum_{i_2} \sum_{i_3} \cdots \sum_{i_n} (\cdot) $$

Now, the value inside the sum factors. Then, you are supposed to know that in such case, in general $\sum_i \sum_j (a_i \,b_j ) = \sum_i (a_i \sum_j b_j ) = ( \sum_i a_i )( \sum_j b_j) $ .