The confidence interval can be derived intuitively by replacing the standardized mean with the standard normal and variance with sample variance, but is there a formal limit? I'm trying to prove if the following is correct:
For $z > 0$, $Z \sim N(0, 1)$, $\mathbf{P} \left( \bar{X} - z \frac{S}{\sqrt{n}} < \mu < \bar{X} + z \frac{S}{\sqrt{n}} \right) \rightarrow \mathbf{P} (- z < Z < z)$ as $n \rightarrow \infty$ (where $S$, $\bar{X}$ are the sample sd and mean for $n$ observations, $X_i$ are iid and the mean and variances exist so CLT can be applied).
So far, I've done the following:
For some RV $Y$,
$\mathbf{P} \left( Y - z \frac{S}{\sqrt{n}} < \mu \leqslant Y + z\frac{S}{\sqrt{n}} \right) =\mathbf{P} \left( - S < \frac{Y - \mu}{z /\sqrt{n}} \leqslant S \right)$. By the strong LLN $S^2 \xrightarrow{p}\sigma^2 \Rightarrow S \xrightarrow{p} \sigma$, hence $S \pm \frac{Y - \mu}{z/ \sqrt{n}} \xrightarrow{p} \sigma \pm \frac{Y - \mu}{z / \sqrt{n}}\Rightarrow S \pm \frac{Y - \mu}{z / \sqrt{n}} \xrightarrow{d} \sigma \pm\frac{Y - \mu}{z / \sqrt{n}}$, hence $\mathbf{P} \left( - S < \frac{Y - \mu}{z/ \sqrt{n}} \leqslant S \right) \rightarrow \mathbf{P} \left( - \sigma <\frac{Y - \mu}{z / \sqrt{n}} \leqslant \sigma \right)$.
But here's where I get stuck, I can't replace $Y$ with $\bar{X}$ as $\bar{X}$ varies with $n$ so there's two limits going on here. How do I proceed?
Outline:
Inetead of $\mathbf{P} \left( - S < \frac{Y - \mu}{z/ \sqrt{n}} \leqslant S \right),$ use $\mathbf{P} \left( - z < \frac{\bar X - \mu}{S/ \sqrt{n}} = Y \leqslant z \right).$
Find the limit of $Y$ using Slutsky's Theorem (see Wikipedia, if not in your text). Use the CLT on $\bar X$ and the fact that $S$ tends to constant $\sigma$. For an approx. 95% CI with sufficiently large $n$ use, $z = 1.96$ from standard normal tables.