Changing Order of Sums of Non-Negatives

Question

I want to prove something and I would like to use the identity: $$\sum\limits_{i=1}^\infty\sum\limits_{n\in A_i}a_n=\sum\limits_{n \in \cup_{i=1}^\infty A_i}a_n$$ Here $A_i$ are disjoint sets of natural numbers and $a_n$ are non-negative real numbers. It seems like the identity should work, but I can't find a good argument for why. It seems rather important that $a_n$ are all non-negative. My question is when can I justify changing the sums like this?

Answer 1

1. Changing the order of terms in a series can sometimes change whether and to what value it converges to.
2. Some series converge to the same value no matter how you rearrange terms. These series are called unconditionally convergent. In symbols: $$\sum_{n=1}^\infty a_n = \sum_{n=1}^\infty a_{\sigma(n)} < \infty$$ for any permutation of indices $\sigma$.

3. It turns out that because the real numbers are complete, every absolutely convergent sequence of real numbers is unconditionally convergent. This means that if the sum of absolute values converges— i.e. $\sum_{n=1}^\infty |a_n|<\infty$, — then the series converges to the same value no matter how you rearrange terms.

4. But a convergent series with no negative terms will always be absolutely convergent, and therefore unconditionally convergent.

5. Next, note that sums like $\sum_{x\in X} p_x$ do not specify an order in which to add terms. As a result, they are not always well-defined (i.e. you can't tell whether they converge, or to what value.)

   For example, suppose a series $\{a_n\}$ converges but not unconditionally, (e.g. if it has some negative numbers in it), and let $A$ be the (multi)set of values in $\{a_n\}$, allowing duplicates, then $$\sum_{a\in A}a$$ is not well-defined because the answer you get, or whether you get a convergent value at all, depends on the order in which you add the values. And sets don't specify an order.

6. A sum like $\sum_{a \in A} a$ has a well-defined value only if it doesn't matter in what order you add up the terms in $A$. In other words, only if every possible way of ordering them yields the same sum.

7. For similar reasons, in your example $A \equiv \bigcup_{i=1}^\infty A_i$ is a set, and therefore the expression $$\sum_{n \in \bigcup_{i=1}^\infty A_i} a_n$$ only has a well-defined value if you get the same answer no matter how you iterate over the set $A$.

8. But all of the numbers $a_n$ are positive. And the series converges when you add it up in the special order $$\sum_{i=1}^\infty \sum_{n\in A_i} a_n$$

   therefore the series converges unconditionally. You can add up the $a_n$ in any order you want, and you can therefore express this as:

   $$\sum_{i=1}^\infty \sum_{n\in A_i} a_n = \sum_{n \in \bigcup_{i=1}^\infty A_i} a_n$$