This question is from Hogg, McKean and Craig's book titled "Introduction to Mathematical Statistics."
3.4.19. Let the random variable $X$ be $N(\mu,\sigma^2)$. What would this distribution be if $\sigma^2 = 0$? Hint: Look at the mgf of $X$ for $\sigma^2 > 0$ and investigate its limit as $\sigma^2 \to 0$.
My Thought Process: Following the hint leads us to an MGF $M_X(t)$ equal to $e^{(\mu t+2 \sigma^2 t^2)}\rvert_{\sigma^2 = 0} = e^{\mu t} $ assuming the pdf has the form $$f(x) = \cfrac{1}{\sqrt{2 \pi}\sigma}e^{\left\{- \frac{1}{2}\left(\frac{x-\mu}{\sigma}\right)^2 \right\}},$$ for $ −\infty < x < \infty.$
The question is that by taking the limit $M_X(t) = E(e^{tX})|_{\sigma^2 \to 0}$ and declaring that the resulting mgf is that of the Gaussian RV when $\sigma^2 = 0$, are we not assuming that the limit and the integral can be interchanged?
Please correct me if I am wrong but $f(x)$ does not converge uniformly to the so called dirac-delta function $\delta(x)$, the characterization of which is pointed out here. So just because the limit of mgf exists does not mean that it corresponds to the pdf of a "degenerate Gaussian RV". I think that for Gaussian RV, the variance has to necessarily be greater than $0$. In other words, I feel that the question is not correct when looked at from a rigorous point of view.
There are threads where people have mentioned that it corresponds to distributions where probability of $X \neq \mu$ is $0$ ie $X = \mu$ almost surely. But that is besides the point. The question asks for the limit of a Gaussian RV as variance approaches $0$ and I think that such a distribution cannot exist.
Another issue with this approach of taking limit of mgf as variance approaches zero is that of the right continuity of the cdf (let's call it $F(x)$). The value of $F(x)$ at $x=\mu$ is $0.5$ but when you take the limit $\sigma^2 \to 0$, it poses a problem as we now need $F(\mu) = 1$ for right continuity of cdf. As pointed out here in the comments by Dilip, right continuity is an issue.
All I want to know is that am I right in thinking that no such "limit" exists for a Gaussian RV as the variance approaches $0$ and that the question is not correct when looked at with a lens of mathematical rigor?
Please provide feedback and critique my thought process.
For convergence in distribution it is not necessary for densities to converge.
If $X \sim N(\mu,\sigma^{2})$ and $Y=\mu$ then $P(|X-Y|>\epsilon) \leq \frac {\sigma^{2}} {\epsilon^{2}} \to 0$ as $ \sigma \to 0$. Hence, $X$ does converge to $Y$ in probability and this also implies convergence in distribution.