How do you state what type of estimator is used when they are the same for a given noise model (maximum likelihood vs weighted least squares)?

43 Views Asked by At

From my understanding, if we are dealing with an estimation problem for which the noise is known to be additive gaussian, the maximum likelihood estimator is the same as the weighted least squares estimator. In this scenario, would it be more reasonable to say that a maximum likelihood estimator was used or that a weighted least squares estimator was used?

On the other hand, when the noise model is not known, can we make an assumption about the noise model and say that we use a maximum likelihood estimator? Or would it be called a weighted least squares estimator since we do not actually know the noise model?

1

There are 1 best solutions below

0
On BEST ANSWER

From my understanding, if we are dealing with an estimation problem for which the noise is known to be additive gaussian, the maximum likelihood estimator is the same as the weighted least squares estimator. In this scenario, would it be more reasonable to say that a maximum likelihood estimator was used or that a weighted least squares estimator was used?

It doesn't matter, since as you say, the two are equivalent in this case.

On the other hand, when the noise model is not known, can we make an assumption about the noise model and say that we use a maximum likelihood estimator? Or would it be called a weighted least squares estimator since we do not actually know the noise model?

You should use the terminology which describes the analysis you performed most faithfully.

A maximum likelihood estimator requires you to make an explicit assumption about the class of distributions from which the error terms are drawn. It can be a much better choice than least squares when the true error distribution is fat tailed; for instance, when the sample size is small and the excess kurtosis is very large, or perhaps in some applications where the error distribution doesn't even have finite fourth or second moments. These cases are rare in regression analysis, however, so people typically don't pay much attention to them.

Otherwise, as long as the class of distributions that you pick for the error term is sufficiently high capacity, using least squares versus using MLE is unlikely to make a big difference. The least squares estimator is unbiased, consistent and asymptotically normal thanks to the central limit theorem (allowing you to get reliable t-statistics) even when the population distribution of the error terms is not normal.

Regardless of which path you chose to follow, you should explicitly inform the reader about what you've done.