If $J:{\Bbb N}\to{\Bbb N}\times{\Bbb N}$ is a bijection, then $\sum_{n=1}^\infty a_{J(n)}=\sum_{i=1}^\infty\sum_{j=1}^\infty a_{ij}$

Question

...assuming that $a_{ij}$ are nonnegative reals and the sum is convergent. Can someone give me a proof or reference for this theorem? I remember reading it in my textbook once, but now I forget and it is not something that can be easily googled, since I don't have a name to go with the theorem.

Answer 1

The expression $\sum_{i,j}a_{ij}$ is defined as the supremum of the finite sums. It easily follows that this expression is at most the (expression on the) right hand side: If $F$ is a finite subset of $\mathbb N\times\mathbb N$, then $\sum_{(i,j)\in F}a_{ij}\le\sum_{i,j\le n}a_{ij}=\sum_{i=1}^n\sum_{j=1}^na_{ij}\le\sum_{i=1}^n\sum_{j=1}^\infty a_{ij}\le\sum_{i=1}^\infty\sum_{j=1}^\infty a_{ij}$.

For the other inequality, verify first that if the right hand side is infinite, so is the left: Either one of the $\sum_{j=1}^\infty a_{ij}$ is infinite, and therefore $\sum_{j=1}^n a_{ij}$ is unbounded as $n\to\infty$. Or each of these sums $b_i$ is finite, but $\sum_{i=1}^\infty b_i$ is not. Fix $M$ and $\epsilon>0$. Pick for each $i$ an $n_i$ such that $\sum_{j=1}^{n_i}a_{ij}>b_i-\epsilon/2^{i+1}$, so (for any $k$) $\sum_{i=1}^k\sum_{j=1}^{n_i}a_{ij}>(\sum_{i=1}^k b_i)-\epsilon$. Now, if $k$ is large enough, the latter sum is larger than $M$.

Assuming now that the right hand side is finite, for any $\epsilon>0$ we can find $n$ such that restricting $i,j$ to vary in $\{1,\dots,n\}$ gives us a sum within $\epsilon$ of the whole sum, and therefore the right hand side is at most the supremum of all these finite sums, which is at most $\sum_{i,j}a_{ij}$.

Similarly, we can verify that $\sum_{n=1}^\infty a_{J(n)}$ also equals $\sum_{i,j}a_{ij}$ by checking that any finite sum $\sum_{(i,j)\in F} a_{ij}$ is bounded by a partial sum $\sum_{n\le k}a_{J(n)}$ and, obviously, all these partial sums are bounded by $\sum_{i,j}a_{i,j}$, by definition.

This result appears, for instance, at the beginning of Tao's book An introduction to Measure theory. It is a particular case of Fubini's theorem, or Fubini-Tonelli. Tao calls it "Tonelli's theorem for series". Rudin also discusses it.