I am trying to solve all of the parts of problem no. 3 given in the following image -
My solution to each part is as follows:-
Every character out of of 44 characters is equally probable.
Let $X$ be an $r.v.$ such that $p(X=x_i) = \frac{1}{44}, i = 1 \ldots 44 $
$H(X) = -\sum_{i=1}^{44}p(x_i)\log_{2}(p(x_i))$
$= -p(x_{1})\log_{2}(p(x_{1})) -p(x_{2})\log_{2}(p(x_{2})) \ldots p(x_{44})\log_{2}(p(x_{44}))$
$= -\frac{1}{44}\log_{2}(\frac{1}{44}) - \ldots -\frac{1}{44}\log_{2}(\frac{1}{44})$
$= -\log_{2}(\frac{1}{44})$
$= \log_{2}(44)$
$= 5.45$Every word out of of 2000 words is equally probable and every word has 4.5 words on average.
Let $Y$ be an $r.v.$ such that $p(Y=y_i) = \frac{1}{2000}, i = 1 \ldots 2000 $
$H(Y) = -\sum_{i=1}^{2000}p(y_i)\log_{2}(p(y_i))$
$= -p(y_{1})\log_{2}(p(y_{1})) -p(y_{2})\log_{2}(p(y_{2})) \ldots p(y_{2000})\log_{2}(p(y_{2000}))$
$= -\frac{1}{2000}\log_{2}(\frac{1}{2000}) - \ldots -\frac{1}{2000}\log_{2}(\frac{1}{2000})$
$= -\log_{2}(\frac{1}{2000})$
$= \log_{2}(2000)$
$= 10.96$
Hence, the entropy per character is: $\frac{H(Y)}{4.5} = 2.44$
Are the above solutions correct? If yes, can someone please help me find the entropy for the 3rd part? Thanks