polynomial curve fitting: higher order models' root mean square error does not decrease

Question

I am trying to fit a curve for 15 data points. I started by creating a linear model and observing the root mean square difference, followed by quadratic, cubic and increasing the degree of polynomial at each iteration and also measuring the root mean square difference for each model. My data points are as follows:

x = {-1.45, -1.12, -0.50, 1.40, 2.28, 3.50, 3.75, 4.50, 5.21, 5.98,6.65, 7.76, 8.56, 9.64, 10.87};
y = {4.30, 4.24, 3.10, 1.50, 5.73, 2.30, 5.31, 3.76, 7.40, 4.13, 9.45, 5.40, 12.67, 8.42, 14.86};

So far the results I got are as follows:

d - the degree of the model. d=2 is quadratic model and d=3 is cubic model

d   rmsd
1   2.42613
2   1.95418
3   1.95373
4   1.91411
5   1.90665
6   1.86912
7   1.81113
8   1.80812
9   1.66018
10  1.36661
11  1.36122
12  1.75445
13  0.993169
14  1.795
15  3.52391

For degree 15, root mean square difference should be closer to zero, regardless of over fitting concerns. Reason being, if you think about it mathematically, it will be fifteen equations and fifteen unknowns and should have a solution. My question is what could have gone wrong in the polynomial curve fitting? I am just testing the linear algebra concepts here. I understand higher degree models may not be right choice.

Answer 1

I tried working out the same as you using Excel. My results agree with yours up to $d=11$. From that point on we begin to differ. The reason is that the difference in size of the values becomes so great that no meaningful analysis is possible. In Excel, by the time you are trying to work with $d=11$ you have to find the 11th power of each x-value and these range from $4.9 \times 10^{-4}$ to $2.5 \times 10^{9}$. Depending on the software you are using to find the fit, it will fail earlier or later, but eventually the results will start to be meaningless.