Does the fact that
$$\int_{-\infty}^{\infty}\exp\left(-\frac{1}{2}x^2\right)\mathrm{d}x=\sqrt{2\pi}$$
Have something to do with the fact that the regularized factorial of infinity is also $\sqrt{2\pi}$?
$$\infty!=\prod_{n=1}^\infty n=\exp\left(\sum_{n=1}^\infty\log n\right)=\exp(-\zeta'(0))=\exp\left(\frac{1}{2}\log2\pi\right)=\sqrt{2\pi}$$
If so, what is the connection between them?
Since $\zeta(s)=\frac{\eta(s)}{1-2^{1-s}}$ and $\eta(0)=\frac{1}{2}$, we have $\zeta(0)=-\frac{1}{2}$. From: $$ \zeta'(s) = \zeta(s)\cdot\frac{d}{ds}\log\zeta(s) = \zeta(s)\left(\frac{\eta'(s)}{\eta(s)}-\frac{2^{1-s}\log(2)}{1-2^{1-s}}\right)\tag{1}$$we get $\zeta'(0)=-\eta'(0)-\log(2)$, and the problem boils down to showing: $$ \eta'(0) = \frac{1}{2}\log\frac{\pi}{2}\tag{2} $$ On the other hand, $\eta'(0)$ is directly related with the Wallis product through its series definition: $$ \eta'(0)=\frac{1}{2}\sum_{n\geq 1}(-1)^{n+1}\log\left(\frac{n+1}{n}\right)=\frac{1}{2}\log\prod_{n\geq 1}\left(1-\frac{1}{4n^2}\right)^{-1}\tag{3}$$ but: $$ \prod_{n=1}^{N}\left(1-\frac{1}{4n^2}\right)^{-1} = \frac{\Gamma\left(\frac{1}{2}\right)^2 \Gamma(N+1)^2}{2\,\Gamma\left(N+\frac{1}{2}\right)\Gamma\left(N+\frac{3}{2}\right)}\tag{4}$$ so by Gautschi's inequality the limit of the RHS of $(4)$ as $N\to +\infty$ is exactly $\frac{1}{2}\,\Gamma\left(\frac{1}{2}\right)^2$, and through the substitution $x=\sqrt{z}$ we have: $$ \int_{0}^{+\infty}e^{-x^2}\,dx = \frac{1}{2}\int_{0}^{+\infty}z^{-1/2}e^{-z}\,dx = \frac{1}{2}\,\Gamma\left(\frac{1}{2}\right).\tag{5}$$
So, a summary of the connection:
The arrows can be reversed as you like. We also have a detour, since the value of $(3)$ is gladly provided by the Weierstrass product for the sine function: $$ \frac{\sin x}{x}=\prod_{n\geq 1}\left(1-\frac{x^2}{\pi^2 n^2}\right) \tag{6}$$ through an evaluation at $x=\frac{\pi}{2}$. No wonder, since $\Gamma(x)\,\Gamma(1-x)=\frac{\pi}{\sin(\pi x)}$.