I am setting up a Fermi Quiz. A quiz where you have to estimate the answers to complicated questions, your answers may be out by orders of magnitude.
A question may be:
"How many piano tuners are there in Chicago?"
The correct answer being around 290, however any answer between 140 and 580 is considered quite close.
What would be the best way to calculate the score for such a quiz.
I have been thinking about this a lot, because i wanted to make something similar.
In my opinion the best method for scoring is based on the answers of the other participants in the quiz.
You get 100 points if the answer is exactly the right answer, otherwise you get 100 points minus the percentage of participants whose answer is better than yours.
An answer $A$ is better than yours if there are less answers between the correct answer and $A$ than between your answer and the correct answer.
This might seem difficult to calculate, but it is quite easy if you have the distribution of answers available.
Example: if the correct answer is $100$, and the set of answers is ${1,15,50,70,80,95,99,101,110,300,1000}$ the scores would be:
$1:0$ points
$15:10$ points
$50:20$ points
$70:40$ points
$80:60$ points
$95:80$ points
$99:100$ points
$101:100$ points
$110:80$ points
$300:60$ points
$1000:40$ points