Basic questions concerning sample means and distributions

Question

I have the following questions:

1. Is The value of the sample mean always the population mean $\mu$, in any sample?
   I am confused about whether or not it is.

2. Is the sampling distribution of the sample mean exactly normal, regardless of the sample size, when sampling from a normal population?

3. Is the effect of increasing the sample size to reduce the standard deviation of the sample mean?

I worked out a few examples, but I'm still not sure.
Any help or guidance would be greatly appreciated.

Answer 1

1. I suppose you mean to ask if the $expected$ value of $\bar X$ is $\mu.$ The answer is Yes, provided $\mu$ exists.

Let $X_1, X_2, \dots X_n$ be a random sample from a population with mean $\mu$. Then $E(X_i) \equiv \mu.$ Then

$$E(\bar X) = E[(1/n) \sum_{i=1}^n X_i] = (1/n) E[\sum_{i=1}^n X_i] = (1/n)\sum_{i-1}^n E(X_i) = (1/n)n\mu = \mu.$$

While we're at it, random sampling implies independence of the $X_i$ so that

$$V(\bar X) = V[(1/n) \sum_{i=1}^n X_i] = (1/n)^2 V[\sum_{i=1}^n X_i] = (1/n)^2\sum_{i-1}^n V(X_i) = \sigma^2/n,$$ provided that the population variance $\sigma^2$ exists.

Note: The second equation requires independence. If $X_1$ and $X_2$ are independent, then $V(X_1 + X_2) = V(X_1) + V(X_2).$ But this does $not$ work without independence. As an extreme example, if $X_1 \equiv X_2$ then $V(X_1 + X_2) = V(2X_1) = 4V(X_1).$

2. Yes.

If $X_1 \sim N(\mu_1, \sigma_1^2)$ and $X_2 \sim N(\mu_2, \sigma_2^2),$ then $X_1 + X_2 \sim N(\mu_1 + \mu_2, \sigma_1^2 + \sigma_2^2).$ Thus adding two normal random variables gives another normal random variable. Also, if $X \sim N(\mu,\sigma^2),$ and $a >0$ and $b$ are real numbers, then $aX + b \sim N(a\mu + b, a^2 \sigma^2).$

In the case of random sampling of two observations from a population, the means and variances are equal: $\mu_1 = \mu_2 = \mu$ and $\sigma_1^2 = \sigma_2^2 = \sigma_2.$ Thus $\bar X = (X_1 + X_2)/2 \sim N(\mu, \sigma^2/2).$ Notice that $V[(X_1 + X_2)/2] = (1/4)V(X_1 + V_2) = (1/4)(2\sigma^2) = \sigma^2/2.$

This generalizes to $n$ independent observations from a normal distribution. So that $E(\bar X) = \mu,\,V(\bar X) = \sigma^2/n,$ and $\bar X \sim N(\mu, \sigma^2/n).$

Note: The Central Limit Theorem says that, for large $n$, $\bar X$ has $approximately$ the distribution $N(\mu, \sigma^2/n),$ even if the data $X_1, X_2, \dots, X_n$ are randomly sampled from a non-normal distribution that has mean $\mu$ and variance $\sigma^2$.

3. Yes. Because each observation has $V(X_i) = \sigma^2,$ whereas the sample mean of $n$ observations has $V(\bar X) = \sigma^2/n.$ So as the sample size $n$ increases, the variance of the sample mean decreases.