The problem I have is about convergence of series expansions of random fields (or stochastic processes, if you will), which don't converge in the norm I want, that is $L_\infty$, but in $L_2$. I have an intermediate result, which could bring me one step further, but I'm not quite sure about it, and would like to know whether the proof I have is ok. Here it goes:
Theorem: Let the random field $r\in L_\infty(\mathcal D\times\varOmega)\subset L_2(\mathcal D\times\varOmega)$, where $\mathcal D$ is a bounded domain in $\mathbb R^d$ and $(\varOmega,P)$ is a complete probability space with probability measure $P$, have an $L_2$-convergent expansion $$r(x,\omega)=\sum_{l\in\mathbb N_0} r_l(x)\xi_l(\omega)$$ with $r_l\in L_2(\mathcal D)$ and $\xi_l\in L_2(\Omega)$ and $E(\xi_l\xi_k)=\delta_{jk}$. Then $r_l\in L_\infty(\mathcal D)$ for all $l\in\mathbb N_0$.
Proof: The proof consists of two parts:
a) First, we need to show that $E(r\xi_l)\in L_\infty(\mathcal D)$ for all $l$. Therefore let $\phi=\xi_l$ and $R=\operatorname{ess sup}_{x \in \mathcal{D},\omega\in\Omega}|r(x,\omega)|$. Then we have \begin{align} \|E(r\phi)\|_{L_\infty(\mathcal D)} &=\operatorname{ess sup}\limits_{x \in \mathcal{D}} \left| \int_\Omega r(x,\omega) \phi(\omega) dP(\omega) \right|\\ &\leq\operatorname{ess sup}\limits_{x \in \mathcal{D}} \int_\Omega \left| r(x,\omega) \right| \left| \phi(\omega) \right| dP(\omega)\\ &\leq\int_\Omega R \left| \phi(\omega) \right| dP(\omega)\\ &\leq\left[ \int_\Omega R^2 dP(\omega) \int_\Omega \phi(\omega)^2 dP(\omega)\right]^{1/2}\\ &=\left[ R^2 \cdot 1\right]^{1/2}=R, \end{align} which proves the assertion.
b) Now we see that \begin{align} E(r\xi_l) &= \int_\Omega r(\cdot,\omega) \xi_l(\omega) dP(\omega) \\ &= \int_\Omega \sum_k r_k(\cdot)\xi_k(\omega) \xi_l(\omega) dP(\omega) \\ &= \sum_k r_k \int_\Omega \xi_k(\omega) \xi_l(\omega) dP(\omega) \\ &= \sum_k r_k \delta_{kl} = r_l \\ \end{align} Together with part a) this proves that $r_l\in L_\infty(\mathcal D)$.
Is this proof correct? I'm always a bit uneasy about proofs in this area, since I tend to overlook subtleties like "can this integration and summation be exchanged" and similar things.
And, if it is correct: has this already been proven somewhere or are there similar results that lead there faster? If the result is correct, I can't quite believe I'm the first to prove this.
a) is ok. But notice that in b) you have $$r(x, \omega) = \left.\left(L_2 \text{-}\lim_{N \to \infty} {\sum_{k=0}^N {r_k(\cdot)\xi_k (-)}}\right) \right|_{(\cdot,-) = (x,\omega)}$$ for $\lambda^d \otimes P$-almost all $(x,\omega)$. There is a subsequence $(N_n)_n$ of $(N)_N$ such that $$r(x, \omega) = \lim_{n \to \infty} {\sum_{k=0}^{N_n} {r_k(x)\xi_k (\omega)}} \quad (1)$$ for $\lambda^d \otimes P$-almost all $(x,\omega)$. Let $\mathcal{N}$ be a measurable $\lambda^d \otimes P$-nullset such that equality holds for all $(x,\omega)$ outside of $\mathcal{N}$. By definition of the product measure the set $\mathcal{N}_x = \{\omega \in \Omega \: | \: (x,\omega) \in \mathcal{N}\}$ is a measurable $P$-nullset in $\Omega$ for $\lambda^d$-almost all $x \in \mathcal{D}$. For such $x$ the equality $(1)$ holds for all $\omega$ outside of $\mathcal{N}_x$, so it holds with probability $1$. So for such $x$ we have $$E(r(x,\cdot) \xi_l(\cdot)) = \int_\Omega {r(x,\omega)\xi_l(\omega)} \mathrm{d}P(\omega) = \int_{\mathcal{N}_x} {\lim_{n \to \infty} {\sum_{k=0}^{N_n} {r_k(x)\xi_k (\omega)}}\xi_l(\omega)} \mathrm{d}P(\omega).$$ (Note that you have to use the same kind of reasoning to explain the third line of a).) Next we have to check that the sequence of functions $\sum_{k=0}^{N_n} {r_k(x)\xi_k (\omega)}$ is uniformly integrable in $\omega$. For that it suffices that its $L_2$-norms are uniformly bounded. They can be calculated: $$\left\|\sum_{k=0}^{N_n} {r_k(x)\xi_k (\cdot)}\right\|_{L_2(\Omega)} = \sum_{k=0}^{N_n} {r_k^2(x)}.$$ Now use the $L_2$-convergence of the series $\sum_{k=0}^N {r_k(\cdot)\xi_k (-)}$. It implies the convergence of the $L_2$-norms $$\left\|\sum_{k=0}^N {r_k(\cdot)\xi_k (-)}\right\|_{L_2(\mathcal{D} \times \Omega)} = \ldots = \int_{\mathcal{D}} {\sum_{k=0}^N {r_k^2(x)}} \mathrm{d}\lambda^d(x)$$ to some finite constant. Fubini's theorem was used in between. In any case, the convergence of the norms is monotone, so they are actually uniformly bounded by the finite constant they converge to. But this means that for $\lambda^d$-almost all $x$, $\sum_{k=0}^N {r_k^2(x)}$ is bounded, uniformly for all $N$. So we get the desired uniform integrability for almost all $x$. This allows the exchange of limit and integral, and you can proceed similarly as you did before.