What is the entropy of a random variable $X$ which denotes the number of rolls of a fair die required to get an outcome less than or equal to $3$

148 Views Asked by At

enter image description here

I know that $H(x) = -\sum(p(x_i)\log(p(x_i))$

And I have found $P(x_i)=1/6\cdot(5/6)^{i-1}$

However I do not know how to calculate this when I put $P(x_i)$ into the formula $H(x)$

1

There are 1 best solutions below

5
On BEST ANSWER

Since it is $\le 3$, $P(x_i)=(\frac{1}{2})^i$. Therefore $H(x)=\sum_{i=1}^\infty (\frac{1}{2})^i \, i \ln{2}=2 \ln{2}$.

Note I used $\ln((\frac{1}{2})^i)=i \ln(1/2)$ with $\ln(1/2)=-\ln 2$ and $\sum_{i=1}^\infty i\,(\frac{1}{2})^i=2$.