I am not understanding the central limit theorem.
From wikipedia:
...suppose that a sample is obtained containing a large number of observations, each observation being randomly generated in a way that does not depend on the values of the other observations, and that the arithmetic average of the observed values is computed. If this procedure is performed many times, the central limit theorem says that the computed values of the average will be distributed according to the normal distribution
what I'm confused about is...if we have a sample of n observed values, then the average of the population will be the sum of all the observed values divided by the total number of observed values. So we will have an average....THE average, meaning ONLY one average, so how can ONE value have a "distribution"? Obviously I'm missing something or interpreting what the definition is saying wrongly, so can somebody help me out?
Edit: Should I think of this as like...let's say we have 1 value. It will have an average. Then we have another value, and take the average of the two values. Then a third value, and find the average of the three. Eventually as you get larger and larger numbers, the "distribution" of all these separate averages will be normal, with the average value eventually equaling the expected value mu?
You have a bunch of data consisting of independent observations or executions of a random experiment. Each of these will be a random variable with a certain distribution (it's not a value, it's some data with a distribution). Each "package" of data, or each random variable has an expected value or an average. What the CLT says is that if the number of random variables (or observations) is very very large (tends to infinity), then the averages of each observation will have a normal distribution. Collecting the averages is like an observation. So these will have a distribution (a normal one). To state it better:
You have random variables $X_1,X_2,X_3,\dotsc,X_n$. Let's define a random variable $$Y_n=\frac{X_1,\dotsc,X_n}{n}.$$ Then $Y_n$ will have a normal distribution as $n\to \infty$. $Y_n$ is not the average of averages, it is a random variable that at least takes the values of the averages of $X_1,\dotsc,X_n$.