Entropy Calculation and derivation of logarithm

129 Views Asked by At

I have probabilities as $$p_1 = 0.4,\ p_2 = 0.3,\ p_3=0.2,\ p_4=0.1$$

hence entropy is given by: $$H(x) = -\big(0.4\cdot \log_2(0.4) + 0.3\cdot \log_2(0.3) + 0.2\cdot \log_2(0.2) + 0.1\cdot \log_2(0.1)\big)$$

I derive this to $$H(x) = -\big(1 - \log_2(10) + 0.3\cdot \log_2(3)\big)$$

and I am unable to derive it further

can you please say if I just need to use calculator or it is possible to use log tricks.

1

There are 1 best solutions below

2
On BEST ANSWER

You could take $$\begin{align}-\big(1 - \log_2(10) + 0.3\cdot \log_2(3)\big) = -\big(1 - \log_2(2\cdot 5) + 0.3\cdot \log_2(3)\big) \\ = -\big(1 - 1-\log_2(5) + 0.3\cdot \log_2(3)\big) \\ =\log_2(5)-0.3\cdot \log_2(3)\end{align}$$ I don't think there is much left you can do with this besides stick it into a calculator