I want to make regression of $Y$ on basis which consists of two sets $\phi_1(X), \phi_2(X), ..., \phi_K(X) $ and $\psi_1(Z),\psi_2(Z),...,\psi_M(Z)$. $X$ and $Z$ are independent random variables
I know that $\psi_1(Z),\psi_2(Z),...,\psi_M(Z)$ have zero mean and are independent from $\phi_1(X), \phi_2(X), ..., \phi_K(X)$
My question is: Instead of solving least squares $\min\sum(y_i - \sum_{k=1}^Ka_k\phi_k(x_i) - \sum_{m=1}^Mb_m\psi_m(x_i))^2$
Can I first make regression $\min\sum(y_i - \sum_{k=1}^Ka_k\phi_k(x_i))^2$
Find the residuals $\epsilon_i = y_i - \hat y_i = y_i - \sum_{k=1}^Ka_k\phi_k(x_i)$
And then make regression of residuals on the second basis set
$\min\sum(\epsilon_i - \sum_{m=1}^Mb_k\psi_k(z_i))^2$
My question is will I obtain the same coefficients for $a_k$ and $b_k$ in both regressions? Intuitively it seems correct, because of zero mean and independence of two basis sets, but how to prove it?
In short, if the functions are orthogonal then $X'X$ is block diagonal matrix, where the first block $X_1'X_1$ correspond to the first $K$ base functions, and the second block $X_2'X_2$ correspond to the second $M$ base functions. Therefore, $(X'X)^{-1}X'y$ will yield the same results as $(X_1'X_1)^{-1}X_1'y$ and for $(X_2'X_2)^{-1}X_2'y$, separately.