I need to approximate a function f, but I cannot do so with frequencies that exceed 1kHz
What is the best approximation I can get? Is taking the Fourier transform then zeroing any term above 1kHz the best approximation? Or can I fiddle with the lower order terms and get a better fit?
This is a made up scenario, but I have to prove the same concept with Walsh transforms. I am fairly certain that the lower order terms form the best approximation from random twiddling and hill climbing searches, but I need proof.
I believe the proof is something very similar to a least squares regression proof, but I can't get it. Has this problem been solved before? At least in the Fourier domain?
The Fourier transform is unitary. Therefore the best fit in the $L^2$ norm in the frequency domain is also the best fit in the time domain. This means that suppressing all frequencies outside the allowed band indeed gives the best approximation.