i.i.d binary random variable question

Question

Suppose there are i.i.d. binary random variables $X_i \sim X$ with distribution $P(X=1) = 0.75$ and $P(X=0) = 0.25$

i) For $n=5$ and $e=0.1$, which sequences fall in the typical set $A_e^n$? What is the probability of $A_e^n$?

ii) How many elements are in the essential bit content set $S_e$ for $X^5$ for $e = 0.1$?

Answer 1

First, compute the entropy:
$H = - \frac{1}{4} \log \frac{1}{4} - \frac{3}{4} \log \frac{3}{4} = 0.561 $ (bits/symbol)

So, $2^{-n H(X)} = 0.1431$

Now, the probability of a given 5-sequence with $k$ ones is $0.75^{k} \times 0.25^{5-k}$ Compute and tabulate this for each $k$, and see which sequences fall in the respective $e-$typical set.

Is the above clear for you? Can you go on from here?