How to choose degree for polynomial regression?

Question

I know how to perform polynomial regression. But is there any method to use for estimating the degree of the polynomial that is best suited? Some kind of meta-regression.

With best suited I mean the grade that has the highest probability of being the true degree of the source for the data.

For example, if we look at this picture we can easily "see" that a polynomial of degree 4 would fit nicely:

A more generalized question is if there is any method to determine if the source is polynomial at all or if it is exponential or something else.

Answer 1

AIC or BIC is the right path to go. These criteria helps you to determine the model that best approximates the generating mechanism. Another, more basic approach is Ramsey RESET test for model misspecification. For example, see here http://lipas.uwasa.fi/~sjp/Teaching/ecm/lectures/ecmc8.pdf

Answer 2

An alternative to polynomial regression is to a fit with Chebychev polynomials, which essentially is a least squares fit.

Usually the coefficients will decrease from the low order terms, and you can stop when the coefficients get small enough.

You can then convert from Chebychev form to polynomial form.

Hope this is hand-wavey enough.