Write and Analytical expression and compute H(X) for $2^{-i}$

37 Views Asked by At

X is a random variable over the positive integers {1,2,3...} such that for each positive integer i,

Pr[X = i] = $2^{-i}$

Write an Analytical expression and compute H(X)

1

There are 1 best solutions below

2
On BEST ANSWER

$$H(X)=-\sum_{i=1}^{\infty}p_i\log_2 p_i=\sum_{i=1}^{\infty}\dfrac{i}{2^i}=\dfrac{\dfrac{1}{2}}{(1-\dfrac{1}{2})^2}=2\ \ bits$$This makes sence and there exist an optimal coding scheme as following:

we code $X=1$ with binary sequence $0$

we code $X=2$ with binary sequence $10$

we code $X=3$ with binary sequence $110$

we code $X=4$ with binary sequence $1110$

we code $X=5$ with binary sequence $11110$

we code $X=6$ with binary sequence $111110$

and so on

this code is also an instantaneous code and as you can see the average length of the former binary code sequences is exactly as much as $H(X)$ (That's why this code is referred to as optimal).