problem in realizing "AEP-like limit"

Question

I have a problem in realizing the solution of this problem:

As I marked in the picture, I cannot understand how $\lim(p(X_1,X_2,\dots,X_n))^{1/n}$ changed to a base $2$ logarithm and then again it changed to what you see in the picture. Would you please explain how and why it happened?

Answer 1

call $Y = p(X_1, \cdots, X_n)^{1/n}$.

By definition : $Y = 2^{log_2(Y)}$.

And $log_2(Y) = log_2(p(X_1, \cdots, X_n)^{1/n}) = \frac{1}{n} log_2(p(X_1, \cdots, X_n))$.

$X_1, \cdots, X_n$ are idependent thus $p(X_1, \cdots, X_n)) = \prod_i p(X_i) \implies log_2(p(X_1, \cdots, X_n)) = log_2(\prod_i p(X_i)) = \sum_i log_2(p(X_i))$.

Sum up all of these facts with remark that the $lim$ operator taken over $n$, you have what happens in the solution.

Answer 2

Just to clarify the above solution a bit more, \begin{align} \lim_{n\to\infty}p(x^n)^{\frac{1}{n}} & = \lim_{n\to\infty}2^{\frac{1}{n}\log p(x^n)}\\ & = \lim_{n\to\infty}2^{\frac{1}{n}\sum_{k=1}^n\log p(x_k)} \\ & = 2^{\lim_{n\to\infty}\frac{1}{n}\sum_{k=1}^n\log p(x_k)} \\ & = 2^{-H(P)} \end{align} where the second line uses the fact that $x_i's$ are i.i.d., the third line uses the continuity of the map $t\mapsto 2^t$, and the last line follows due to strong law of large numbers that: $$ \frac{1}{n}\sum_{k=1}^n \log p(x_k) \xrightarrow{\text{a.s.}}\mathbb{E}[\log p(x)] = -H(X) $$ as $n\to\infty$, almost surely.