Unbiased Estimator - Single Observation

Question

Here is a statement that I was told to be true :

"We have a random sample of n observations X1, X2, . . . , Xn, and let Xi be a single observation from the sample. Then, Xi is an unbiased estimator of the population mean."

Is this statement true because the expected value of that single observation will be the same as the population mean? If so...then is this implying that the single observation you picked is a random sample itself? I guess what I am confused about is how does just a single number's expectation have the exact value of the population mean?

Answer 1

Is this statement true because the expected value of that single observation will be the same as the population mean?

Yes.

If so...then is this implying that the single observation you picked is a random sample itself?

Yes. All $X_1$ to $X_n$ are random samples from the looks of it.

I guess what I am confused about is how does just a single number's expectation have the exact value of the population mean?

Ummmm, why can't it be that way? :)

More seriously, I understand your confusion about this matter. By itself, I wouldn't call this paragraph the most mathematically unambiguous. The terminology I would use is as follows. There is a probability distribution $P$ and $X_1 \ldots X_n$ are i.i.d. samples drawn from $P$. In this case, each $X_i$ is an unbiased estimator of the mean of $P$. This is obviously because each $X_i$ is a random variable drawn from $P$, and by definition, $E[X_i]$ is the mean (say $\Theta_P$), so we have that the bias of the estimator $X_i$ is $E[X_i - \Theta_P] = 0$.

We also have that the empirical mean, i.e. $\frac{X_1 + \ldots + X_n}{n}$ is an unbiased estimator of the mean of the population. This means that the average of the $n$ samples is an unbiased estimator of the population mean. This about it this way. If you just took three samples, say $X_1$, $X_2$, $X_3$, the mean $\frac{X_1 + X_2 + X_3}{3}$ is also obviously an unbiased estimator of the population mean. The same thing holds if you took just two samples. Why would it stop holding all of a sudden if you took just one sample? It won't stop holding! Thus the empirical mean when you have just one sample, i.e. $\frac{X_1}{1}$ is also an unbiased estimator.

Now instead of taking just one sample $X_1$, what's the harm in taking n samples $X_1, \ldots, X_n$ and focusing on one particular sample in this list $X_i$? Obviously nothing wrong. That is what is happening in your problem.

P.S. All this holds because I assumed the samples were drawn i.i.d. from the distribution. In plain English, this means that there is nothing statistically different between sample $X_1$ and sample $X_2$, $X_1$ and $X_3$, and so on. Thus taking some $X_i$ is the same as taking $X_1$ or $X_n$ or any other sample.

Answer 2

Your confusion may be reduced if you are careful to distinguish random variables from numbers. The quotation does not mix this up, but your wording does. The usual convention is to use capital letters for random variables, $X_i$, for example. Its value is unknown (at least to you), but you may know its distribution or perhaps its expectation. On the other hand if an observation is revealed, then use lower case notation and think of this as a regular number, for example $x_i=42$ for $i=3$.