Like the title says, I'm trying to find
$$\int_0^r x^2 e^{-x^2}\,dx$$
Where $r$ is some finite value.
I've done one step using integration by parts with $u=x^2$ and $dv=e^{-x^2}dx$, which has left me with
$$\left.x^2 \int_0^x e^{-t^2}\,dt\,\right|_0^r - \int_0^r 2x \int_0^x e^{-t^2}\,dt\,dx$$
and more questions than answers. I'm aware that the integrals in $t$ are a scaling of the error function, and don't have a representation in elementary functions. For the second term in this I think I should try another integration by parts, but then I would need the integral of the error function, and am not sure how to get that either.
Am I going about this the right way? What do I do next?
One can solve the integral using a nice little trick (often called Feynman integration). We generalize the problem by adding a free parameter to the exponential (the reason we do this is that $\frac{d}{dt} e^{-tx^2} = -x^2 e^{-tx^2}$ which for $t=1$ is the integrand we are trying to evaluate). We start by defining the function
$$f(t,r) \equiv \int_0^r e^{-tx^2}{\rm d}x$$
Now observe that
$$\frac{\partial f(t,r)}{\partial t} = -\int_0^r x^2 e^{-tx^2}{\rm d}x \implies \int_0^r x^2 e^{-x^2}{\rm d}x = \left[-\frac{\partial f(t,r)}{\partial t}\right]_{t=1}$$
Substituting $y = \sqrt{t}x$ we can evaluate $f$ in terms of the error function as $$f(t,r) = \frac{\sqrt{\pi } \text{erf}\left(r \sqrt{t}\right)}{2 \sqrt{t}}$$
and by differentiating and taking $t=1$ we get the result $$\int_0^r x^2 e^{-x^2}{\rm d}x = \frac{1}{4} \sqrt{\pi } \text{erf}(r)-\frac{1}{2} e^{-r^2} r$$