Desirable properties of statistical estimators?

Question

What are some of the properties that people will consider when designing a statistical estimators? For example, unbiasedness and sufficiency are some of the factors considered.

Please give some factors in your answer and include formal details, derivations, and examples.

(I would rather ask this question here since Cross Validated seems to be on the applied side but not on the theoretical side and will not explain the terminologies of statistical distributions in detail.)

Answer 1

A statistical estimator is just a random variable for what we can measure. We hope this measurement is reliable, and so anything that means the probability distribution is "well-behaved" is a desirable property. Therefore we would want things like:

• Small variance for the estimator. If we know that the estimator has a large variance, that means that taking the mean of the estimator is likely not a good estimate - we could be far off!
• Robustness. Sometimes we have outliers in our data. Do these outliers screw up our estimate? Robustness is a measure for how well an estimator can deal with outliers.
• "Applicability". How applicable is this estimator to reality? In our derivation we do things like assume normality or some other distribution. Non-parametric estimators do not require you assume a particular distribution, which can be a desirable property if you plot your data and know it doesn't follow known distributions. Many times this just means relaxing some assumptions. Popular tests like the Wilcoxon Ranked Sum test use estimators which don't require normality, only that the distribution is symmetric, which better matches the data and thus is more realiable.
• Bias. Of course you want an unbiased estimator since that means that as you get more data your estimate converges to the "real" value. However, there is a trade-off because many times biased estimators can have a lot less variance and thus give better estimates when you have less data. One well-known example is Ridge Regressions.
• Anything else that makes sense. For example, you'd want less skew in the estimator, since a heavily skewed estimator might get the mean right but you won't know if your estimate is far off in the tail. If you think of something that makes sense, there's probably a paper about it.