What's the best estimate from very few observations?

Question

My actual problem, I have three sensors, $A$, $B$ and $C$ that all measure the same physical property and that I believe can all be considered as independent random variables and identically distributed, maybe Gaussian.

So these three sensors may for example have different biases (from the true value). Given that there are only three measurements, what is the best way of combining the three measures to get a best estimate of the true value? I can't believe that averaging would be a good choice since one measure (which might be considered an 'outlier') could be far away from the other two which might be closer together.

The median measure might be a good choice, but in the example above it essentially throws away the outlier. Shouldn't that measure have some weighting in this case?

What is the best way to combine these measurements to achieve a best estimate of the true value? Or do I need more information or assumptions?

Answer 1

You need to figure out (a) whether differences among A, B, and C are due to random variation among independent, identically distributed measurements or (b) whether one or more of A, B, and C is systematically biased (so that perhaps A tends systematically to give a higher reading than B, or C.) You say that A, B, and C are identically distributed, but it sounds as if you don't really believe it.

If it's (a), and if readings are indeed normally distributed, then the average (or 'mean') is best. If the readings happen to be distributed according to a Laplace distribution, the median would be best.

You would need to do an experiment to determine whether it's (b). You could have all three sensors measure the same input in several dozen experiments. Then use an ANOVA to see if there are systematic differences among readings of A, B, and C. If so, perhaps physically adjust the sensors to read the same. Or if adjustment is impossible, at least you would know how to adjust numerical outputs from each sensor before averaging the three.

---

Note: The distribution theory for medians of normals may be beyond what you are able to do. But for situation (a), a simple simulation gives an idea that, for normal data, the mean of three is, as you suspect, a more stable (less variable) summary than the median of three.

For a particular input, suppose that A, B, and C give measurements distributed as $\mathsf{Norm}(\mu = 50, \sigma = 3).$ Let's simulate 10,000 runs with a measurement from each sensor. Then elementary statistical theory says that the standard deviation (SD) of the means of three is $\sigma/\sqrt{3} = \sqrt{3} = 1.73.$ The simulation agrees with that and approximates the SD of the medians as 2.02, which is noticeably larger.

Code for the simulation in R is shown below, in case it is of interest;

m = 10^4;  n = 3;  x=rnorm(m*n, 50, 3)
MAT = matrix(x, nrow=m)
a = rowMeans(MAT);  h = apply(MAT,1,median)
mean(a); mean(h)
## 49.9931   # aprx expected means = 50
## 49.97891  # aprx expected medians = 50
sd(a); sd(h)
## 1.735878  # aprx SD of means = 1.732
## 2.022162  # aprx SD of medians

Note: About 200 years ago Gauss and Laplace reportedly had 'discussions' whether normal or Laplace distributions were more appropriate for modeling various astronomical and physical phenomena. A roughly comparable simulation for Laplace data gave the SD of means as about 1.36 and the SD of medians as slightly smaller 1.33.

Answer 2

This is the typical problem of data validation and reconciliation.

Let $y_i$ the measurement given by sensor $i$ and $\sigma_i$ its corresponding standard deviation. So, for the case of $n$ sensors (each of them providing one measurement), you need to minimize $$F(Y)= \sum_{i=1}^n\frac {(y_i-Y)^2}{\sigma_i^2}$$ Differentiate and set $F'(Y)=0$. Solving for $Y$ (the most probable value of the measurement), you should get $$Y=\frac{\sum_{i=1}^n \frac{y_i}{\sigma_i^2} } { \sum_{i=1}^n \frac{1}{\sigma_i^2} } $$