I'm looking for a way of doing polynomial regression myself (I mean without the help of Excel or other cool software like Mathematica, ...).
I've heard of algorithm doing that, like Gauss-Newton Algorithm and Least Squares Algorithm, but I couldn't find an easy/straightforward way of using these algorithms.
I'm just asking if there is a nice way of computing them following a step by step method, like for the linear regression.
It is linear least squares with a kernel built of monomials.
$${\bf c_o}=\min_{\bf c}\|{\bf \Phi c - d}\|_2^2$$
where
$${\bf\Phi} = [{\bf 1,x,x^2,\cdots,x^n}]$$
where the 1,x etc are column vectors with the values and $\bf c$ is a column vector linearly combining them together (the vector of coefficients) and $\bf d$ is the vector of data points to fit to (function values).