Using SLLN To Estimate A Probability

40 Views Asked by At

Suppose we flip a coin to figure out the probability p to get head. We flip $n$ times and get the frequency $\hat{p}$.

By strong law of large number, $P(\lim_{n \rightarrow \infty} \hat{p}=p)=1$, so I know $\hat{p}$ is a good estimator for p. However, I don't know how many times I need to flip.

To say in a mathematical way, for a given $\epsilon$, is there a way to find a $n$, such that $P(|\hat{p}-p|<\epsilon)=1$?

1

There are 1 best solutions below

0
On BEST ANSWER

Unfortunately, the only $\epsilon$ that is guaranteed to work here is $\epsilon=1$ since you are estimating the probability of heads. What you are looking for is called a confidence interval (CI) for $p$. A confidence interval built from a random variable (read: a estimator before we collect data) $\hat{p}$ is $CI_c(\hat{p}):=(\hat{p}-a,\hat{p}+b)$ has confidence level $c\in[0,1]$ if $P(p \in I_c(\hat{p})) \geq c$

Once you collect data, you will have a specific value for $\hat{p}$ and the constructed interval either does or does not contain $p$, but if $c$ is close to $1$ then you know you had a high a-priori probability of the interval covering $p$.

There are a number of approaches to constructing a confidence interval for $\hat{p}$. The most straightforward is the Wald Interval:

$$\text{Wald}CI_c := \hat{p} \pm z_{1-c/2}\sqrt{\frac{\hat{p}(1-\hat{p})}{n}}$$

Where

  • $z_{1-c/2}$ is the $1-c/2$ percentile of the standard normal distribution
  • $\hat{p} = \frac{n_{heads}}{n}$

Now the main choice for you is to pick a confidence level $c \in (0,1)$ that you can live with. :)