Averaging estimates of probability

58 Views Asked by At

Say one weather forecaster says there is an 80 percent chance of precipitation at a given place and time.

Another forecaster says there is a 30 percent chance.

I have no reason to think one forecaster is better than another. What is my best guess at the correct percent chance of precipitation?

Can I just take a simple arithmetic average: 55 percent? Or do I need a different kind of mean?

1

There are 1 best solutions below

0
On

As the comments say, there is not enough information to answer.

However. Say we can trust that both are, in our estimation, good forecasters. But, what does really that mean? There are many ways of assesing the quality of forecasts. In this context, it can be acceptable (but by no means necessary) to adopt this criterion: to trust a forecaster each time she says "There is a probability $p$ of rain" is equivalent to believe that the following game is fair: "I bet $1$ dollar that it will rain; I win $\frac{1-p}{p}$ if it does, I lose it if not". You can check that the expected net win is zero.

Then, in this scenario, we can imagine that the two forecasts, with estimations $p_1$ $p_2$, trigger two bets. If it rains, I will win $$\frac{1-p_1}{p_1}+\frac{1-p_2}{p_2}$$

and I will lose $2$ if not. This is, in my estimation, a fair bet. But this is equivalent to a single bet with probability $p$ such that

$$ \frac{1-p}{p}=\frac{1}{2} \left(\frac{1-p_1}{p_1}+\frac{1-p_2}{p_2}\right)$$

or

$$ p=\frac{2 p_1 p_2}{p_1+p_2}= \frac{2}{1/p_1 + 1/p_2} $$

Hence, the equivalent probability (single equivalente forecast) is the harmonic mean of each individual probability.

In your example: $p = 43.6\%$

One objection against this proposal: this is not invariant wrt negation. If instead of "probabilty of rain" with speak of "probability of no rain", the equivalent "average" probability is not one minus the original one.


An alternative to deal with the last objection:

A forecast with probability $p$ is associated with a bet that wins $a$ or lose $b$. We have the restrictions $a p - b (1-p) = 0$ (fair bet) and $a b = 1$ (invariance upon complement), which gives $a=\sqrt{(1-p)/p}$ and $b=\sqrt{p/(1-p)}$

By the same reasoning, the equivalent probability now gives

$$p_{eq} = \frac{\sqrt{p_1 p_2}}{\sqrt{p_1 p_2}+\sqrt{(1-p_1)(1-p_2)}}$$

sort of a normalized geometric mean.

In your example: $p_{eq} = 56.7\%$