In what cases is it possible to exchange between Series and Integral signs, namely performing the following operation?
$$\sum\int F(x) \text{d}x ~~~~~ \to ~~~~~ \int\sum F(x) \text{d}x$$
What are the sufficient / necessary conditions? I know it might be a hard work to write here the inherent theory but it would really help me to understand once for all that question.
In a general manner, consider $\big( f_n \big)_{n \in \mathbb{N}}$ a series of continuous functions such that $f_n \to f$ uniformly in a compact interval $[a,b]$ . In your example, \begin{equation} f_n = \sum\limits_{i = 1}^n F_n(x) \end{equation} represent the partial sums with $f_n(x) \to \sum\limits_{i = 1}^{\infty} F_n (x) = F(x)$.
We want to prove that: \begin{equation} \lim_{n \to \infty } \int\limits_{a}^{b} f_n(x) dx = \int\limits_{a}^{b} \lim_{n \to \infty } f_n (x) dx = \int\limits_{a}^{b} f (x) dx \end{equation}
Proof We will show that for all $\epsilon > 0 $ exists $N \gg 0$ such that: \begin{equation} \left|\int\limits_{a}^{b} f (x) dx - \int\limits_{a}^{b} f_N (x) dx \right| < \epsilon. \end{equation}
Notice that: \begin{equation} \begin{aligned} \left|\int\limits_{a}^{b} f (x) dx - \int\limits_{a}^{b} f_N (x) dx \right| \leq \int\limits_{a}^{b} \left|f_N(x) - f(x) \right| dx \end{aligned} \end{equation}
Because $f_n$ is uniformly convergent, there exists $N$ such that $|f_N(x) - f(x)| < \epsilon_1 $ Therefore:
\begin{equation} \int\limits_{a}^{b} \left|f_N(x) - f(x) \right| dx < \int\limits_{a}^{b} \epsilon_1 = (b-a)\epsilon_1 , \quad \forall x \in [a,b]. \end{equation}
By making $\epsilon_1 = \epsilon /(b-a)$ we obtain that:
\begin{equation} \left|\int\limits_{a}^{b} f (x) dx - \int\limits_{a}^{b} f_N (x) dx \right| < \epsilon. \end{equation}
This proves the theorem.