Weighted Average from Errors

Question

This should be a really easy problem, but I want to know the right way to do it instead of making something up.

Data:

Given the data set ( the unit is nano meters ): $$ D = \{ 3386, 3290, 3372, 3450 \} $$ The following is the respective errors: $$ \sigma_D = \{ 50, 180, 42, 100 \} $$

Question:

How do I find the weighted average?

Answer 1

On the contrary, this is a hard problem. You need to have a model for the distribution of the measurement errors. If they are normally distributed, so your measurement of $3386 \pm 50$ represents a normal curve with mean $3386$ and standard deviation $50$ you should weight each measurement with the inverse of its error squared. This will give you the most probable value. Unfortunately, the normal distribution is often a terrible model as the tails are so small while measurements that are a ways off are rather common. If the measurements are like a meter stick, so the correct value has a uniform distribution through the range mean-error to mean+error you can just find the interval consistent with all the measurements. Here the minimum is $3350$ coming from the last measurement and the maximum is $3414$ coming from the third.

Answer 2

This problem is related to data reconciliation.

The most probable value $Y$ is given by the minimization, with respect to $Y$, of

$$\Phi=\sum_{i=1}^n \left(\frac {Y-D_i}{\sigma_i} \right)^2\implies \frac{d\Phi}{dY}=0\implies Y=\frac{\sum_{i=1}^n \frac{D_i} {\sigma_i^2} }{\sum_{i=1}^n \frac{1} {\sigma_i^2}}$$ Applied to the given numbers $Y=3382$.

You could be interested by this paper.

Answer 3

Adding to Claude Leibovicis answer I'd like to contribute how to calculate the error of the weighted average (can't comment unless 50 reputation…) using the notation above: $$ \Delta Y = \pm \frac{\sqrt{\sum_j (p_j \sigma_j)^2}}{\sum_j p_j} $$ with weight $p_j = max_i(D_i)/\sigma_j$.

A more practical way to calculate both is then $$ \Delta Y^2 = \left( \sum_j \sigma_j^{-2} \right)^{-1} \, ,\\ Y = \Delta Y^2 \sum_j \frac{D_j}{\sigma_j^2} \, . $$

This was adapted from a undergrad physics laboratory introduction booklet of TU Bremen (Germany). The ressource is not available online.