I am looking at second order polynomial approximation, specifically at this link. However, I am stuck in the one dimensional case: I do not think I can calculate the following:
$$ X^{T}*X^{-1}*X^{T}*y$$
Suppose I have as sample points $x = [1.0,2.0,3.0,4.0,5.0]$, then from what I understood in the link my matrix should become: \begin{bmatrix} 1 & 1 & 1 \\ 1 & 2 & 4 \\ 1 & 3 & 9 \\ 1 & 4 & 16 \\ 1 & 5 & 25 \end{bmatrix} Ignoring the columns $x_1*x_2, \cdots x_{nx-1}x_{nx}$ because I do not have more dimensions. Suppose I calculate $X^{-1}$ with Matlab pinv command, still I cannot do $X^T * X^{-1}$. Is this a problem with the single dimension and it gets solved with $d > 1$?
Looks like a typo on the site,
$$ \boldsymbol{\beta} = \color{red}{(} {\bf X}^T {\bf X}\color{red}{)^{-1}} {\bf X}^{T} {\bf y} $$
If you use
y = [0.0, 1.0, 1.5, 0.5, 1.0], you should getbeta = [-0.9, 1.22142857, -0.17857143]