I have an unfair coin with unknown bias $p$ chosen uniformly at random from $[0, 1]$. I keep tossing the coin multiple times and log the outcomes.
It is intuitive to see I gain more information about $p$ with each toss. But what might be an information theoretic approach to represent the situation that scales with the number of tosses?
The fraction of heads you get is a measure of $p$. In the normal approximation, valid if you have a bunch of tosses and $p$ is not too close to $0$ or $1$, the standard deviation of the number of heads in $N$ tosses is $\sigma=\sqrt{Np(1-p)}$. This gives an indication of how accurately you know $p$