Information theory: typical set proof

110 Views Asked by At

I'm trying to understand this proof from my information theory class. It's a proof of the property of the typical set.

Prove the following:

$$ 2^{-n(1+\epsilon)H(X)}\leq p(x^n) \leq 2^{-n(1-\epsilon)H(X)}\quad \text{and} \quad \displaystyle 2^{-n(1+\epsilon)H(Y)}\leq p(y^n) \leq 2^{-n(1-\epsilon)H(Y)}$$

Proof: Note that $$\begin{align} \frac{1}{n}\log\left(\frac{1}{p(x^n)}\right) &=\sum_{i=1}^n \frac{1}{n}\log\left(\frac{1}{p_X(x_i)}\right) \\ &= \sum_{x\in \chi} \frac{1}{n}N(x\vert x^n)\log\left(\frac{1}{p_X(x_i)}\right) \tag{1} \end{align}$$ For $x^n \in \mathcal{T}_\epsilon^{(n)}(X)$, we have $$ (1-\epsilon)P_X(x)\leq \frac{1}{n}N(x\vert x^n)\leq(1+\epsilon)P_X(x),\quad x\in\chi \tag{2} $$ Combining (1) and (2) shows that $2^{-n(1+\epsilon)H(X)}\leq p(x^n) \leq 2^{-n(1-\epsilon)H(X)}$ By symmetry, we have $\displaystyle 2^{-n(1+\epsilon)H(Y)}\leq p(y^n) \leq 2^{-n(1-\epsilon)H(Y)}$

I understand how they reach eq 1 and 2 but I don't really understand how eq 1 and 2 completes the proof as the last line says.Can anyone give me a pointer on where to start?

Thanks a lot!