Confusion about the import of the standard error

Question

The standard error is defined: $$ SD(\bar{X}) = \frac{\sigma}{\sqrt{n}} $$ where $\sigma$ is the standard deviation of the population and $n$ is the sample size. In the book I'm reading, it seems to be saying the $SD(\bar{X})$ should be as small as possible, but shouldn't it rather be as close as possible to the actual standard deviation $\sigma$, rather than approaching zero? How have I misunderstood the what the standard error tells us?

Answer 1

There are two different quantities:

• standard error: variance of the estimator that serves to estimate the mean. $V(\hat{X}) = nV(X)/n^2=V(X)/n$ therefore $\sqrt{V(\hat{X})} = \sigma/\sqrt{n}$ as you have corectly indicated. This number tends to 0 as $n$ grows to infinity (this is the law of large numbers)
• population variance: the variance of the samples that can be estimated through a different estimator, say, $\hat{V}$. This random variable, $\hat{V}$ is what you would like to approximate the actual variance. Alternatively, $\hat{D}$ could represent an estimator for the standard deviation, and you would like it to approximate the standard deviation, not zero, as you correctly pointed out.

An estimator is a function of the random variables, so it is also a random variable. The estimator has a mean and a variance. In general, you want the mean to be close to the value you want to estimate, and the variance to be as close as possible to zero.

Examples of estimators pointed out above

• $$\hat{X}= \sum X_i/n$$

• $$\hat{V} = \sum ( X_i-\hat{X})^2/n $$ or $$\hat{V} = \sum ( X_i-\hat{X})^2/(n-1) $$

• $$\hat{D} = \sqrt{\sum ( X_i-\hat{X})^2/n} $$ or $$\hat{D} = \sqrt{ \sum ( X_i-\hat{X})^2/(n-1) } $$