how do i find the variance of an estimator?

Question

If the Estimator was simply the sample mean $s=\frac{\sum{x}}{n}$ taken from a binomial distribution (a random example) how would i calculate the variance of this? I am trying to use the difference between the expectations squared but im not sure what the expectation of the infinite sum would be.

Answer 1

If $X_1, X_2, \dots, X_n$ is a random sample from a population with mean $\mu$ and variance $\sigma^2,$ let $T = \sum_{i=1}^n X_i.$

Then

$$E(T) = E\left(\sum_{i=1}^n X_i\right) = \sum_{i=1}^n E(X_i) = \sum_{i=1}^n \mu = n\mu.$$

Also, elements of a random sample are independent, so we have

$$V(T) = V\left(\sum_{i=1}^n X_i\right) = \sum_{i=1}^n V(X_i) = \sum_{i=1}^n \sigma^2 = n\sigma^2.$$

Also, with $\bar X = \frac{1}{n}\sum_{i=1}^n X_i = \frac{1}{n}T,$ so that

$$E(\bar X) = E\left(\frac{1}{n}T\right) = \frac{1}{n}E(T) = \frac{1}{n}n\mu = \mu.$$

Thus. the expected value of the sample mean $\bar X$ is the population mean $\mu.$ (We say that $\bar X$ is an unbiased estimator of $\mu.)$

Moreover,

$$V(\bar X) = V\left(\frac{1}{n}T\right) = \left(\frac{1}{n}\right)^2V(T) = \left(\frac{1}{n}\right)^2n\sigma^2 = \frac{1}{n}\sigma^2 = \sigma^2/n.$$

Notes: (1) In the first displayed equation the expected value of a sum of random variables is the sum of the expected values, whether nor not the random variables are independent.

(2) However, the variance of the sum of random variables is not necessarily equal to the sum of the variances, unless the random variables are independent.

[As a trivial case, if all $n \ge 2$ of the $X_i = X,$ then the $X_i$ are not independent and we have $V\left(\sum_{i=1}^n X_i\right) = V(nX) = n^2V(X) \ne nV(X).$ As another example, if $X_1 = -X_2$ with $V(X_1)=V(X_2) > 0,$ then $V(X_1+X_2) = V(0) = 0 \ne V(X_1)+V(X_2).]$

(3) For the standard deviation of the mean of a random sample, we can take square roots to get, $SD(\bar X) = \sigma/\sqrt{n}.$ (Sometimes this is called the 'standard error' of $\bar X.)$