Where to begin self-learning of numerical approximation algorithms?

Question

I would like to better my skills in numerical analysis and specifically that part of numerical analysis which deals with approximations of functions of all kind. Since I don't know the nomenclature that well yet I unfortunately cannot formulate my main goal with appropriate terms. But in essence I would like to build up a toolbox which enables me to write excellent approximation algorithms, and to understand their possible limitations.

Concretely I am asking for reference to a book/course which covers approximation algorithms and their error analysis with some major programming language, such as C++. While many such courses do exist, I am asking for the one you would recommend to a friend or a colleague who is just starting to learn the applications of numerical analysis, and would like to have a comprehensive guide for it.

Answer 1

I like “Approximation Theory and Approximation Practice” by Trefethen. Clear writing, great references, lots of pictures. The first six chapters are available on-line. The code is written using Matlab. The approximation techniques are implemented in a Matlab add-on package called Chebfun.

If you insist on using C++, you could try this package, instead.

Answer 2

The first result I found immensely helpful for numerical analysis was Taylor's theorem, with special attention towards approximating or bounding the remainder. Once you've got that, you can build up all sorts of interesting error bounds for constructing approximate derivatives, integrals, and interpolants.

I enjoyed Numerical Analysis by Gautschi. In addition to the stuff above, that book also introduces the reader to (the massive field of) approximating solutions to differential equations.