What is $$\int xe^{-x^2} dx\quad?$$
I used substitution to rewrite it as $$\int -\dfrac{1}{2}e^u\, du$$ but this is too hard for me to evaluate. When I used wolfram alpha for $\int e^{-x^2} dx$ I got a weird answer involving a so called error function and pi and such (I'm guessing it has something to do with Euler's identity, but I'm fairly certain this is above my textbook's level).
Your substitution was spot on. You put $$u = -x^2 \implies du = -2x\;dx \implies x\,dx = -\frac 12\, du$$
And having substituted, you obtained $$\int -\dfrac{1}{2}e^u\, du$$ Great work. But I think you gave up too early!: $$-\frac 12\int e^u \, du = -\frac 12 e^u + C,\tag{1}$$ and recall, $u = f(x), \;du = f'(x)\,dx$, so $(1)$ is equivalent to $$-\frac 12 \int e^{f(x)}\,f'(x)\,dx = -\frac 12 e^{f(x)} + C$$
So we can integrate as follows, and then back-substitute: $$\int -\dfrac{1}{2}e^u\, du = -\frac 12 \int e^u \,du = -\frac 12 e^u + C = -\frac 12 e^{-x^2} + C\tag{2}$$
Now, to remove all doubts, simply differentiate the result given by $(2)$: $$\frac{d}{dx}\left(-\frac 12 e^{-x^2} + C\right) = -\frac 12(-2x)e^{-x^2} = xe^{-x^2}$$ which is what you set out to integrate: $\color{blue}{\bf x}e^{-x^2} \neq e^{-x^2}$