Minimum number of bits required for transmission?

777 Views Asked by At

A communication company uses a system to transmit four different symbols ${S_1, S_2, S_3, S_4}$. Each symbol has a probability to occur according to the following table \begin{equation*} \begin{array}{c|c|c|c|c} & S_1 & S_2 & S_3 & S_4 \\ \hline p_i & 0.05 & 0.61 & 0.27 & 0.07 \end{array} \end{equation*} Calculate the entropy of the system and the minimum number of bits required for transmission.

So I already calculated the entropy. Regarding the minimum number of bits required for transmission at first I thought "this is straight forward" and calculated

\begin{equation} \lceil \log_2 4 \rceil = \lceil 2 \rceil = 2 \end{equation}

as we have four symbols but now I am not so sure anymore because I didnt take the probalities into account at all. I am afraid I misunderstood the minimum number of bits required to represent four symbols as the the thing I am actually asked to calculate here.

I am sorry in advance if this is super trivial. I just started with information theory and so on and am very new to this.

So how do I calculate the minimum number of bits required for transmission?