I was wondering what the antiderivative of $e^{-x^2}$ was, and when I wolfram alpha'd it I got $$\displaystyle \int e^{-x^2} \textrm{d}x = \dfrac{1}{2} \sqrt{\pi} \space \text{erf} (x) + C$$
So, I of course didn't know what this $\text{erf}$ was and I looked it up on wikipedia, where it was defined as:
$$ \text{erf}(x) = \dfrac{2}{\sqrt{\pi}} \displaystyle \int_0^x e^{-t^2} \textrm{d}t $$
To my mathematically illiterate mind, this is a bit too circular to understand. Why can't we express $\int e^{-x^2} \textrm{d}x$ as a 'normal function'? Also, what is the use of the error function?
When $f$ is a continuous function on the interval $[a,b]$, we can find a function $F$ defined on $[a,b]$ such that $F'(x)=f(x)$ for all $x\in[a,b]$. This is the “fundamental theorem of calculus”; just consider $$ F(x)=\int_{a}^{x} f(t)\,dt $$ There are other functions with the same property, precisely those of the form $F(x)+c$ where $c$ is a completely arbitrary constant.
Sometimes this function can be expressed with the so-called “elementary functions”, that is, polynomials, rational functions, exponential, logarithm, trigonometric functions and any algebraic combination thereof. Some (actually many) functions do not admit an antiderivative expressible in this form; it's the case of $e^{-x^2}$ and it can be proved, although not easily.
Think of a simpler example: if all we have available as “elementary functions” are polynomials or, more generally, rational functions, the function $1/x$ wouldn't admit an “elementary antiderivative”, but it still would have one: $$ \int_{1}^{x}\frac{1}{t}\,dt $$ Since this is a “new” function, we give it a name, precisely “$\log$” and we have extended the tool set. The same happens with “$\operatorname{erf}$”, which has many uses in probability theory and statistics, being related to normal distributions.