I have a practical problem.
I have continuous functions defined on [-1..1] with [-1..1] bound. The functions might have infinite derivatives like $\sqrt x$ has at 0.
One example:
I want to approximate those functions.
Sorry for non-mathematical descriptions, but here are the desirable properties of the approximation system:
- The approximation function should be easily evaluated at given x.
- More parameters - better approximation (in $L_2$ or $L_1$ norm sense)
- Diminishing returns - each parameter improves the approximation less than the previous one did
- The domain of each parameter is some real interval (maybe infinite) and each combination of parameters define a valid approximation function.
- It would be best if the range of parameters is finite though.
- The approximation function should not be [much] more complex than the original function (say, in terms of the number of critical points). I fear that truncated Taylor or Fourier series would exhibit such problem.
I could have used something like a 2D Bezier curve for approximation, but not all parameter combinations define a valid function.
P.S. I'm not actually interested in the approximation per se, but rather in an opposite problem: Defining a continuous parameter space that contains points representing approximations of elementary functions and their combinations.

The first candidate for the "space" you're referring to would be $P_n$ -- the space of polynomials of degree at most $n$ on $[-1,1]$. When constructing approximations, the best basis of this space is typically the set of Chebyshev polynomials. You get better and better approximations by including Chebyshev polynomials of higher and higher degree. One problem, though -- polynomials don't work very well when the approximant has a vertical slope. So, you may have to use rational functions, instead.
There is a nice piece of software called chebfun that does a very good job of constructing approximations using polynomial and rational functions. Some examples here.
All of this is discussed in great detail in an excellent book by Trefethen called "Approximation Theory and Practice". Some parts of it are available free on the chebfun web site.