The Karhunen-Loeve decomposition takes an $L^2$-integrable stochastic process $X_t$ and decomposes it into a series of deterministic processes $e_i(t)$ that form a basis of $L^2$, multiplied by random variable "coefficients" $Z_i$, where these coefficients are determined by an inner product-like integral $Z_i = \int_a^b X_t e_i(t)dt$.
This looks fairly obviously like a generalized Fourier series, with the stochastic processes forming a Hilbert space, but with two key differences:
- The basis elements are specific types of elements in the (possible) Hilbert space, rather than just any set of stochastic processes that form a basis in $L^2$.
- The "coefficients" in this case are random variables rather than scalars, which would only create a correspondence between the K-L decomposition and generalized Fourier series if the underlying vector space of stochastic processes had random variables as its scalar elements.
Are these differences explained simply by non-intuitive methods in the way the underlying vector space of stochastic processes is constructed, or is the K-L decomposition a similar but truly different construction to a generalized Fourier series?
You should look first at the finite dimensional case.
Let $v$ be any $\Bbb{R}^n$-valued random variable with finite mean and variance. Let $\mu= \Bbb{E}[v]$ and $w=v-\mu$. The covariance matrix $$R_{ij} = \Bbb{E}[w_i w_j]$$ Then $R$ is real symmetric thus it diagonalizes into an orthonormal basis $$R^\top = R \implies R = P D P^\top$$ (where $P$ is a real orthonormal matrix and $D$ is a real diagonal matrix)
Moreover $$a^\top R a= \Bbb{E}[(\sum_{i=1}^n a_i w_i)^2]\ge 0 \implies D_{ii}\ge 0$$ Let $$u = P^\top w,\qquad \Bbb{E}[u u^\top]=\Bbb{E}[P^\top u u^\top P]= D_{ij}$$ Whence $$v = \mu+P u$$ where $u$ is a $\Bbb{R}^n$-valued random variable whose components are pairwise non-correlated.
If a process $X$ can be well-approximated by some finite dimensional random variables then this kind of decomposition extends to $X$.