I face a strange problem.
I have data $[X,Y]$ with no noise. If I plot $Y$ as a function of $X$, it looks like a straight line and the regression coefficient is quite good.
For the simplest model $$Y = 1 + aX\\ a = 0.328841508328634 ± 2.36619297413024\cdot 10^{-5}\\ \chi^2/doF = 0.00149066816255509\\ R^2 = 0.999325489519206$$
The problem is that, if I continue adding powers of X, the fit is better and better, all parameters stay highly significant.
However, whatever can be the degree of the polynomial, the residuals keep a sinusoidal shape. This probably reveals that I am using a wrong function. Can this symptom on residuals give an idea of a better function?