approximating data using orthogonal polynomials

Question

Recently, during numerical analysis lecture, I've learned about finding best approximation of element $f$ from normed linear space $E$, defined by $$\inf_{g\in \textbf{G}} \lVert f-g\rVert$$ where $G$ is a subspace of $E$. In particular I've learned about case when $E=C[a, b]$, and $G = \Pi_n$ (polynomials of degree at most n), with inner product defined as $$\langle f, g\rangle = \int_a^bw(x)f(x)g(x)dx$$ for some weight function $w(x)$. I know that if we are given $n$ orthonormal elements $\{g_1, \dots, g_n\}$, then best approximation for $f$ in subspace $G := span\{g_1, \dots, g_n\}$ is $$g := \sum_{i=1}^n\langle f, g_i\rangle g_i$$ Using that, we can approximate continous functions using polynomials. Now inspired by that, I wanted to solve another problem: given $n$ points $(x_i, y_i)$ I want to find polynomial approximation fiting this data. Taking $E=C[a, b]$, and $G = \Pi_n$ and using some weight function $w$, what I would like to do, is set inner product to: $$\langle f, g\rangle = \sum_{i=1}^nf(x_i)g(x_i)w(x_i)$$ because then $$\lVert f-g\rVert^2 = \sum_{i=1}^n[f(x_i)-g(x_i)]^2w(x_i)$$ Which summs errors in points of interests, so best approximation would minimize those. I could then take any function $f(x)$ for which $f(x_i) = y_i$, use Gram-Schmidt method to find orthonormal polynomials, and use those to find best data fit using polynomial of $n$th degree. The problem is, my inner product is not well defined, because $\langle f, f\rangle = \sum_{i=1}^nf(x_i)^2w(x_i)$ which equals to zero for infinitely many functions. To me it feels, like they do not matter in this contex and all of them can be associated with zero function, but i dont know how to fix my choice of spaces/inner product definitions to make it work. Am I going in the right direction?

Answer 1

You need to restrict your degree further, so that the system $p(c,x_i)=y_i$ becomes square or over-determined. Then the zero problem does not exist, at least in the polynomial space. The input data is just a sequence of values.

In some sense one could say that "over-determined" is given for the integral version, where you could approximate the interval with a dense countable subset.

The result for the sum version is then weighted polynomial regression, a well-established tool in time series approximation.