Simulating this $X$ gives very close adherence to a normal distribution, but I can't find a good justification. I've tried 3 things that I couldn't get to work:
- Various CLT arguments.
- MGFs.
- normal-inverse-Wishart conjugate prior.
I'm quite certain that $X$ is not normally distributed, so I'm not looking for a proof, just some justification. Also I'm using the notational convention $\text{Var}(X|Y) = |Y|$.
Intuitively, this is because $Y = \mu + “\mathcal{O}(\sigma)”$ for large $\mu$ and fixed $\sigma$. This suggests that the distribution of $X$ is close to $\mathcal{N}(\mu, |\mu|)$, explaining why $X$ looks like normally distributed.
To give the proof, let us consider the normalized random variable $ Z = \frac{X - \mu}{|\mu|^{1/2}} $. Then the PDF of $Z$ can be computed by
\begin{align*} f_Z(z) &= |\mu|^{1/2} f_X(\mu + z |\mu|^{1/2}) \\ &= |\mu|^{1/2} \mathbf{E} \biggl[ \frac{1}{\sqrt{2\pi |Y|}} e^{-(\mu + z |\mu|^{1/2}-Y)^2/2|Y|} \biggr] \end{align*}
Writing $Y = \mu + \sigma \xi$ for $\xi \sim \mathcal{N}(0, 1)$, this further reduces to
\begin{align*} f_Z(z) &= \mathbf{E} \biggl[ \frac{1}{\sqrt{2\pi |1 + (\sigma \xi/\mu)|}} e^{-[z - (\sigma \xi / |\mu|^{1/2})]^2 / 2 \lvert 1 + (\sigma \xi/\mu) \rvert } \biggr]. \end{align*}
With $\sigma$ fixed and as $|\mu| \to \infty$, we can show that this converges to
\begin{align*} \mathbf{E} \biggl[ \frac{1}{\sqrt{2\pi}} e^{-z^2/2} \biggr] = \frac{1}{\sqrt{2\pi}} e^{-z^2/2} = \phi(z), \end{align*}
the PDF of the standard normal distribution. Therefore $Z$ converges in distribution to $\mathcal{N}(0, 1)$ as $|\mu| \to \infty$.