Why The Population Mean = Sampling Distribution Mean

Question

I'm currently learning about basic statistics and the sampling distribution. You'll have to forgive my misuse of terms as I'm still learning. My question is about how is the average of a population the same as the average of the sampling distribution.

I'll go through my understanding and have done a basic example to try and illustrate, please correct me if I'm wrong.

My understanding is if I have a set of values below - the population:

Set Of Values - Population

Then given a sample size, the sampling distribution is a histogram of the averages of all possible combinations able to be drawn from the population.

For example with a sample size of 2 using the set of values above as the population there are total of 6 possible samples able to be drawn. With a sample size of 3 there are 4 possible samples able to be drawn, as shown below:

Combinations Able To Be Drawn

I can see that the mean of all the sample means is the same as the mean of the population. I know this is the basis of many of the other statistical concepts. But my question is how does this work?

I'm aware this may be a little ahead of my ability to understand, but wondered if someone could help trace through how the mean of a set of values is the same as the mean of all the possible sample means.

Many Thanks

Nick

Answer 1

Firstly note that your sampling is not a random sample which is usually meant in statistical context. Your population consists of 4 groups with the sizes $n_1, n_2, n_3$ and $n_4$. In case of $n=3$ you draw the whole population of 3 groups and you draw no element of the remaining group. This you do for all possible combinations of the groups: $(1,2,3);(1,2,4);(1,3,4);(2,3,4)$ There is no randomness when you draw the elements. Then the calculation of the group mean of the population is the mean of the group-combinations.

$$\overline x_g=\frac14\left(\frac{n_1+n_2+n_3}{3}+\frac{n_1+n_2+n_4}{3}+\frac{n_1+n_3+n_4}{3}+\frac{n_2+n_3+n_4}{3}\right)$$ $$=\frac14\left(\frac{3n_1+3n_2+3n_3+3n_4}{3}\right)=\frac14\left(\frac{3\cdot (n_1+n_2+n_3+n_4)}{3}\right)=\frac14\left(n_1+n_2+n_3+n_4\right)$$

The latter term is the same calculation as in the case of $n=1$ (first table in your picture). Summing up all 4 groups and then dividing the sum by $4$.

Answer 2

For small samples of small populations, you can work out examples as shown in the calculation you linked to, and you can verify for yourself that the mean of all sample means is the population mean. But I'll try to give a "deeper" explanation based on principles of probability.

This is an application of linearity of expectation, which is a very useful fact to know about.

The members of a sample of size $N$ are random variables $X_1,X_2,X_3,\ldots,X_N,$ each of which has the same distribution. This is true even if the population is small and sampling is done without replacement, provided that $X_N$ exists at all (in other words, the population size must be at least $N$ so that it is possible to take such a sample). For a finite population sampled without replacement, first arrange the entire population in a random permutation, and then $X_i$ is the $i$th element of the permutation. By symmetry, the distribution of each $X_i$ is the same. (This is analogous to taking a well-shuffled deck of cards and sampling it by drawing a card from somewhere in the deck.)

In particular, $X_1$ has the same distribution as the population, therefore so does each $X_i,$ and therefore the expected value of $X_i$ is the same as the population mean. Suppose the population mean is $\mu.$ Take $$Y = X_1+X_2+X_3+\cdots+X_N.$$

Then by linearity of expectation, \begin{align} EY &= EX_1+EX_2+EX_3+\cdots+EX_N \\ &= \overbrace{\mu+\mu+\mu+\cdots+\mu}^{\text{$N$ times}} \\ &= N\mu. \end{align}

Note that linearity of expectation does not depend on the variables being independent, so it is OK if they are actually samples from a finite population (and therefore not independent).

The sample mean is $$\frac 1N(X_1+X_2+X_3+\cdots+X_N) = \frac1N Y$$ and its expectation is $E\left(\frac1N Y\right) = \frac1N(EY) = \mu,$ the same as the population mean.