I am trying to dilute $N$ bits of information in a randomized message of size $M$.
Given that I know the location of the bits I want to conserve the most. The way I picture this possible is to take each individual bit $b$ and dedicate to him $p$ bit with $p < 1$ known. So I want to get a Bernoulli trial with probability $x$ such that it would take $1/p$ trials to retrieve the bit $b$.
To rephrase my question and give examples I need to find the odd $x$ of flipping $b$ such that:
- When I want to dedicate 1 bit to $b$, $x = 0$.
- When I want to dedicate 0 bit to $b$, $x = 0.5$.
- When I want to dedicate $0.5$ bit to $b$, it would take two samples to retrieve $b$. (Beyond a reasonable doubt, let's say at 95% certainty)
In that way I could distribute "flipping factor" to each of the $M$ bit of the message I want to send and get a message of size $M$ containing $N$ bits.
I have tried to read up on information theory and entropy of the binomial distribution. I get the feeling entropy as a concept describes the inverse behaviour I want. For example lower probability messages transmits the most information (and have the most entropy) whereas here, if a message has perfect 50/50 probability I want it to transmit no information ! I am not well versed in the subject so it is possible I just don't connect the dots.
Thank you