X is a random variable over the positive integers {1,2,3...} such that for each positive integer i,
Pr[X = i] = $2^{-i}$
Write an Analytical expression and compute H(X)
X is a random variable over the positive integers {1,2,3...} such that for each positive integer i,
Pr[X = i] = $2^{-i}$
Write an Analytical expression and compute H(X)
Copyright © 2021 JogjaFile Inc.
$$H(X)=-\sum_{i=1}^{\infty}p_i\log_2 p_i=\sum_{i=1}^{\infty}\dfrac{i}{2^i}=\dfrac{\dfrac{1}{2}}{(1-\dfrac{1}{2})^2}=2\ \ bits$$This makes sence and there exist an optimal coding scheme as following:
this code is also an instantaneous code and as you can see the average length of the former binary code sequences is exactly as much as $H(X)$ (That's why this code is referred to as optimal).