I found a document with a long list of proofs of Gaussian integral converging to $\sqrt \pi$. Interestingly it did not contain a proof by Taylor expansion. Let's try...
$$\int_0^\infty e^{-x^2}dx=\text{lim}_{y\rightarrow \infty}\int_0^y \sum_{n=0}^\infty \frac{(-1)^nx^{2n}}{n!}dx$$
We can exchange the sum and integral by Fubini as $e^{x^2}$ converges for finite integral.
$$=\text{lim}_{y\rightarrow \infty} \sum_{n=0}^\infty\int_0^y \frac{(-1)^nx^{2n}}{n!}dx$$ $$=\text{lim}_{y\rightarrow \infty} \sum_{n=0}^\infty \frac{(-1)^ny^{2n+1}}{n!(2n+1)}$$
And we are looking to prove that this is equal to $\sqrt{\sum_{n=0}^{\infty}\frac{(-1)^n}{2n+1}}=\frac{\sqrt \pi}{2}$. To make things easier we can square both sides. Then, for all $\epsilon$ we need $n_0(k),n_1$
$$|\sum_{n=0}^{m}\frac{(-1)^n}{2n+1}- (\sum_{n=0}^{m} \frac{(-1)^n k^{2n+1}}{n!(2n+1)})^2|<\epsilon, \space \space \space \space \space \space n_0<m,n_1<k$$ $$\Leftrightarrow |\sum_{n=0}^{m}\frac{(-1)^n}{2n+1}- \sum_{i=0}^{m}\sum_{j=0}^{m} \frac{(-1)^{(i+j)} k^{2(i+j)+1}}{i!j!(2i+1)(2j+1)}|<\epsilon, \space \space \space \space \space \space n_0<m,n_1<k$$
... and here I get stuck for now. Can it be done?
Edit: Solution seems to be approximately in range of $n_0(k)=3^k$, and $n_1=ceil(\frac{1}{\epsilon})$.
There are many different reasons why this idea won't work. First of all observe that actually you can NOT interchange the integral sign with the power series: $$\lim_{y\rightarrow+\infty}\int_{0}^{y}\sum_{n=0}^{+\infty}\frac{(-1)^{n}x^{2n}}{n!}dy$$ it is actually a finite value, but here is the problem: the generic term of the sum is not a $L^{1}$ function on $(0,+\infty)$, i.e. the integral: $$\int_{0}^{+\infty}\Bigg|\frac{(-1)^{n}x^{2n}}{n!}\Bigg|dx$$ is NOT finite, thus the equality between the integral of power series and the series of integral does not hold (you can't even hope to apply dominated convergence theorem).
Nice idea, but unfortunately won't work.