Standard error. Why does it get more accurate as N gets bigger?

Question

I came across this quote about the Standard Error in the OpenIntro Statistics book:

We required a large sample in Chapter 4 for two reasons:
1. The sampling distribution of x tends to be more normal when the sample is large.
2. The calculated standard error is typically very accurate when using a large sample.

So what should we do when the sample size is small? As we’ll discuss in Section 5.1.1, if the population data are nearly normal, then x ̄ will also follow a normal distribution, which addresses the first problem. The accuracy of the standard error is trickier, and for this challenge we’ll introduce a new distribution called the t-distribution.

I thought the standard error, defined as $ SE = \frac{s}{\sqrt{n}}$ when we don't know $\sigma$ (the population standard deviation), just gets smaller as n gets bigger. What does it mean that it gets more accurate when using a large sample?

I mean, I read about the CLT and this:

Conditions for x ̄ being nearly normal and SE being accurate...Important conditions to help ensure the sampling distribution of x ̄ is nearly normal and the estimate of SE sufficiently accurate:

• The sample observations are independent.
• The sample size is large: n > 30 is a good rule of thumb.
• The population distribution is not strongly skewed. This condition can be difficult to evaluate, so just use your best judgement.

It seems like the reason to use a large sample is because since the standard error, which is a measure of the uncertainty associated with the point estimate, depends on $s$. And I guess $s$ gets more accurate of the population standard deviation as n increases? Is that right?

I guess this quote is enlightening:

There is one subtle issue in the equation for SE: the population standard deviation is typically unknown. You might have already guessed how to resolve this problem: we can use the point estimate of the standard deviation from the sample. This estimate tends to be sufficiently good when the sample size is at least 30 and the population distribution is not strongly skewed.

So that's it right? As n, the sample size gets bigger, we can be more sure that the sample's standard deviation more accurate reflect the population standard deviation. So s becomes a more reliable estimate as n gets bigger... and so SE, the sampling distribution of the mean, also becomes more accurate (which means... a true estimate of the real standard error). Is what I wrote correct?

Answer 1

Yes, you are correct, and that's the intuition.

And all discussion in the question is fundamentally backed by the following three theorems:

Weak Law of Large Numbers: Suppose the r.v. $\{X_i\}$ are i.i.d. and $\mathbb EX_1^2<\infty$. $S_n=\sum_{i=1}^nX_i$. Then $S_n/n\rightarrow \mathbb EX_1$ in probability, as $n\rightarrow \infty$.

---

Strong Law of Large Numbers: Suppose the r.v. $\{X_i\}$ are i.i.d. with $\mathbb EX_1 < \infty$. Let $S_n=\sum_{i=1}^nX_i$. Then $S_n/n \rightarrow \mathbb EX_1$ almost surely, as $n\rightarrow \infty$.

---

Central Limit Theorem: Suppose the r.v. $\{X_i\}$ are i.i.d. with mean zero and variance one, and $S_n=\sum_{i=1}^nX_i$. Then $S_n/\sqrt{n}$ converges weakly to a standard normal random variable, as $n\rightarrow \infty$.

Answer 2

You are correct.Namely, the larger the sampling size, the smaller the error.

Since $SE = \frac{s}{\sqrt{n}}$ decrases as $n$ grows, by taking a large enough $n$ the error term can be made arbitrarily small.

Answer 3

Yes, the key that you originally may not have understood is that last "subtle issue." Note that the unbiased estimator of $\sigma$ is the sum of the squares of the differences between the samples and the mean of the samples, divided by $(n-1)$ rather than $n$. The distribution of this estimator, which of course average to the actual $\sigma$, is given by the $t$ distribution, which of course gets more peaked as $n$ grows large.

The consequence of using an inaccurate $\sigma$ when making a confidence level statement is that you might think you are quoting a $\pm2\sigma$ range, but are really quoting (say) a $2.2\sigma$ range. For large $n$, this danger is small since the estimate of $\sigma$ is fairly accurate.