I have real data $x(t)$ which I model using the linear regression model: $\displaystyle f(t)= \sum_{i=0}^n c_i \phi_i(t)$, where $c_i$ are the regression coefficients I'm trying to find and $\phi_i(t)$ are a basis function (I chose Legendre Polynomials for their orthogonality).
I noticed something strange I couldn't explain i.e. when I fix $n=1$ for my model I get $\hat{c}_0, \hat{c}_1$, normal! Then when I fix $n=2$ for my model I get $\hat{c'}_0=\hat{c}_0, \hat{c'}_1=\hat{c}_1, \hat{c'}_2=0$ (Seems like $n=2$ didn't add anything to the data modeling!)
What is more strange is that when I fix $n=3$, I get $\hat{c''}_0, \hat{c''}_1, \hat{c''}_2, \hat{c''}_3$ that are completely different (no $\hat{c''}_2=0$)
Does anyone understand what is happening here?
This also happens for other even index coefficients $c_4, c_6, \dots$ i.e. when they are the latest regression coefficient in my model they are null but when I add another odd index coefficient they are not. Also I noticed that the coefficient values I obtain are high (as if to indicate an overfitting!)
The Legendre polynomials are orthogonal, but your regression results (by changing) show that their values on your data have nonzero covariance.
If $f(t)$ is odd that would explain the odd/even pattern with the $c_i$, where the even degree $c_{2n}$ contribute nothing on their own, but are involved in isolating the contribution of odd powers of $x$.