I recently came cross a question related to best-fitting function subspaces as follow.
Let $L^2[-1,1]$ be the Hilbert space of real valued square integrable functions on $[-1,1]$ equipped with the norm $\|f\|$= $\sqrt{\int_{{-1}}^1 {|f(x)|}^2 dx}$.
Obviously, $1,x,x^2,\cdots,x^{n-1}$ are all in $L^2[-1,1]$ for any positive integer $n$.
My question is that for any fixed positive integer $k$($k<n$), are there $k$ functions $h_1,h_2,\cdots,h_k$ in $L^2[-1,1]$ that minimize \begin{equation} {\rm dist}\left({\rm span}(h_1,\cdots,h_k);1,x,\cdots,x^{n-1} \right) := \inf_{\alpha_{ij} \in \mathbb{R} :0\leq i \leq n-1, 1\leq j \leq k} \sum_{i=0}^{n-1} \left\| x^i - \sum_{j=1}^k \alpha_{ij} h_j \right\|^2. \end{equation} If there are $k$ such functions, what are their expressions?
Indeed, the question is equivalent to find a $k$-dimension best-fitting function subspaces of $1,x,x^2,\cdots,x^{n-1}$ in $L^2[-1,1]$.
I feel like this question might be related to principal component analysis(PCA), but I don't know how to generalize PCA to $L^2[-1,1]$.
Does anyone have a reference or a solution which answers this question? Thanks in advance.
First you can see that the infimum is attained with $h_1, \dots, h_k\in \operatorname{span}(x^1, \dots, x^{n-1})$. Indeed, consider $h_1, \dots, h_k$ in $L^1([-1,1])$. Then $$\left\|x^i-\sum_{j=1}^k \alpha_{ij}h_j\right\| \ge\left\|x^i-\pi\left(\sum_{j=1}^k \alpha_{ij}h_j\right)\right\| =\left\|x^i-\sum_{j=1}^k \alpha_{ij}\pi\left(h_j\right)\right\|,$$ where $\pi$ is the orthogonal projection of $\operatorname{span}(1, x, \dots, x^{n-1}, h_1, \dots, h_k)$ on $\operatorname{span}(1, x, \dots, x^{n-1})$.
This implies that the infimum is attained with $h_1, \dots, h_k$ in $\operatorname{span}(1, x, \dots, x^{n-1})$. Then you're left with a finite dimensional problem. By imposing without loss of generality that $\|h_i\|=1$ for all $i$, and showing that the distance is continuous, by compactness it follows that the infimum is indeed a minimum.
I believe you could also compute directly the polynomials for which this minimum is attained.