Expectation value of an estimator

109 Views Asked by At

In wikipedia the following example is given about what an estimator is and what the estimand is: "For example, the sample mean is a commonly used estimator of the population mean. "

estimator=sample mean

population mean= the estimand

value of the estimator = estimate

Now, what does it mean to find the expectation value of the estimator?

If the estimator, in the above example is the sample mean, then when we want to find the expectation value of the estimator, we want to find the expectation value of the mean? What does this even mean? How can we find the expectation value of the mean of a sample?

1

There are 1 best solutions below

5
On BEST ANSWER

The expected value is with respect to the sampling distribution of the sample mean. To explain what that is, fix some population, and imagine you could repeatedly sample from that population. For each sample, calculate the sample mean. Then you get some distribution of sample means which is called the sampling distribution. The expected value of the sample mean with respect to that distribution is what you seek.

Specifically, suppose we draw data independently $y_1, \ldots, y_n$ from a normal distribution with mean $\mu$ and variance $\sigma^2$. We know $\sigma^2$, but don't know $\mu$, and want to estimate it. Then a natural estimator of $\mu$ is the sample mean $\bar{x} = \frac{1}{n}\sum_{i = 1}^n x_i$. What is the distribution of this estimator? Well, since each $y_i$ is drawn from a normal distribution, the sample mean is a sum of normally distributed random variables, which is itself normally distributed. But with what mean and variance? The expected value is

\begin{align} \mathrm{E}(\bar{x}) &= \mathrm{E}\left(\frac{1}{n}\sum_{i = 1}^n x_i \right)\\ &= \frac{1}{n}\sum_{i = 1}^n \mathrm{E}(x_i) \\ &= \frac{1}{n} n \mu = \mu. \end{align}

Note that in the last line I used the fact that the draws are identically distributed. So the expected value of the estimator is the population mean. This is good because in expectation, as we sample over and over again, the sample mean will give us the correct answer. The variance is

\begin{align} \mathrm{Var}(\bar{x}) &= \mathrm{Var}\left(\frac{1}{n}\sum_{i = 1}^n x_i \right)\\ &= \frac{1}{n^2}\sum_{i = 1}^n \mathrm{Var}(x_i) \\ &= \frac{1}{n^2} n \sigma^2 = \frac{\sigma^2}{n}. \end{align}

Thus, $\bar{x} \sim \mathcal{N}(\mu, \frac{\sigma^2}{n})$. This is the sampling distribution of the sample mean. Note that in the second line I used that the draws are independent, and in the third line that they are identically distributed.

Note that not all estimators have the property that their expected value is equal to the population parameter you are trying to estimate (take for example the estimator that always outputs the number $1$). The estimators that do have this property are called unbiased.

Hope this helps.