As per central limit theorem(CLT) $$ \lim_{n \to\infty}P\left(\frac{a\sigma}{ \sqrt {n}}<A_n-\mu<\frac{b\sigma}{\sqrt{n}}\right)=\frac{1}{\sqrt{2\pi}}\int_{a}^{b}e^{-t^2/2}dt $$ where $$ A_n=\frac1n(X_1+X_2+...X_n)\\ \mu=E(X_i)\\ \sigma^2=E((X_i-\mu)^2)<\infty $$
Here $X_i$ are independent random variables.
My understanding so far was that in the limit of large $n$, saying $A_n-\mu$ is distributed as a normal distribution, $N(0,1)$, was same as saying that $A_n$ was distributed as $N(\mu,\sigma/\sqrt{n})$ but the latter would tend to $N(\mu,0)$ or a spike about $\mu$. In other words $\lim_{n\to\infty}A_n\to\mu$ . Is this line of reasoning correct?
I am getting really confused regarding this: "under large $n$, the arithmetic mean,$A_n$, would tend to the expectation,$\mu$". If this was true, there would be no distribution of $A_n$ at all, just a spike at $\mu$. On the other hand, the RHS of CLT indicates that the probability of $A_n-\mu$ lying close to zero is zero..
I have looked at texts and wiki but only got more confused. I hope someone can resolve this stupid doubt.
Yes, the limit of $A_n$ as you define it will be $\delta$-distributed and it can only have one value, its expectation, $\mu$. The right way to do it is to define $$B_n =\frac{1}{\sqrt{n}}(X_1+\dots+X_n)-\sqrt n\,\mu$$ and then it will be distributed as $\mathcal N(0, \sigma)$.
Alternatively, if you like to be able to think in terms of $A_n$, you can interpret the statement as: for large $n$, $A_n$ is approximately distributed as $\mathcal N(\mu, \sigma/\sqrt n)$. It's distribution gets narrower as $n$ gets larger.
The moral of the story is that when you add up random variables, the mean grows linearly with $n$, but the fluctuations grows with $\sqrt n$. So if you divide by $n$, the fluctuations disappear, and the mean approaches the expected value.