everyone. I've been thinking of this problem when reading papers about Fourier series. I think I can state my question as follows:
in the interval $[0, 1]$, I want to approximate an unknown continuous function with maximum frequency $f_0$ with mean $0$ (don't worry about the boundaries). I know it can be represented losslessly by weighted sum of sine waves. However, if I am only allowed to use two functions in $[0,1]$, ( any continuous functions), What they should be so that I can minimize the $L_2$ error? i.e. if the two optimal functions are $f_1$ and $f_2$, the unknown function is $f$, I want to minimize, the expectation of the following, you can assume the occurrence of any functions is 'uniform'.
$$\min_{a_1, a_2}\int_0^1 (f-a_1 f_1 - a_2 f_2)^2 \,dx $$ where $a_1$ and $a_2$ are real numbers.
I am also wondering if there is an information theory viewpoint, those two optimal functions can be considered as the most expressive ones under the constraints. And therefore minimize the conditional entropy of the unseen function?
This seems to be a quite general and easy question, I doubt it is still open, Can someone point me to the right direction? Is there a well-established theory for such approximation? I am not sure if the conditions are strong enough, please make any assumptions as you like.
Thanks, any input will be appreciated!
There is a general theory known as the Hilbert spaces H and the problem you are tackling is known as the closest element in a closed subspace $M \subset H $.