This question arises from a discussion on How to deal with the integral of $e^x e^{x^2}$? where the error function is invoked and there is a comment that this might not be a satisfactory solution (at least for some) because it is more a renaming exercise than a solution. Similarly for the Lambert W, which appears regularly.
Now there are functions we all know and learn, like the trigonometric functions, exponential function and logarithm, which emerge reasonably naturally in various contexts early in our mathematical experience. Hyperbolic functions (sinh/cosh etc) are not so immediately used, but emerge through being the odd and even parts of the exponential function as well as in other ways.
Then there are special functions of various kinds like Bessel Functions, Legendre Polynomials, Hypergeometric functions as well as those in the presenting discussion.
It seems to me that whether we actually like particular functions or not - or consider them as solutions rather than renamings - might depend on the fluency and intuition we have developed in understanding them. It seems to me to be a bit like solving equations by radicals, which is a nice thing to do, because we know what radicals are - but practically we return to numerical estimates very often, so the radical gives an illusion of precision rather than a practical solution.
So my question is approximately "what makes a function familiar enough to be a special function?"
Now this could be too vague, or opinion-based, or more suitable for meta - but on the other hand I thought the discussion was interesting enough to highlight, and maybe others will have some useful insight.
We use the term elementary function for any function that can be expressed in terms of a finite composition of polynomials, exponentials, and inverses of the same (roughly speaking). So it includes logs, and roots, trig and hyperbolic by combining exponentials (and passing to complex numbers), and inverses of those, etc. The way I explain it to students is, “If you can evaluate it on a standard scientific calculator, it's an elementary function.” I second the recommendation of Brian's article.
This is a pretty good place to draw the line if you're teaching an undergrad calculus course. Other functions can be “admitted” when their need comes up in more advanced courses.