The additivity axiom of probabiity

Question

One of the axioms of probability is:

If the sample space is finite and $A$ and $B$ are disjoint events then $Pr[A\cup B]=Pr[A]+Pr[B]$, and if the sample space is infinite, then for for any (possibly infinite number of) disjoint events, like $A_1,A_2,\ldots$, then $Pr[\cup_iA_i]=\sum_iPr[A_i]$

I cannot understand why the first one cannot imply the second one. I have seen several probability books and they have mentioned it and skipped its detail. Can anyone give an example that the union of just two disjoint sets does not imply the union of any (possibly infinite) number of disjoint events?

Answer 1

First of all, there's no reason to restrict the definition of finite additivity to finite spaces. It's just that if a finitely additive measure is defined on a finite space, then, trivially, it's countably additive.

We can extend finite additivity as you've defined it by induction. Your axiom implies that, for any $n \in \mathbb{N}$, if $A_1,...,A_n$ is a sequence of pairwise disjoint events, then $$\sum_nP(A_n) = P(\cup_n A_n).$$

But finite additivity does not imply countable additivity. Indeed, consider the "fair integer lottery" (the uniform distribution over integers) that was of interest to De Finetti. This is a finitely additive probability measure defined on all subsets of $\mathbb{Z}$ such that $P(\{ z\})=0$ for all $z \in \mathbb{Z}$. Countable additivity fails because $$P(\mathbb{Z}) = 1 \neq 0 = \sum_z P(\{z \}).$$

Note that this is analogous to Lebesgue measure on $[0,1]$, for which we have countable additivity but not uncountable additivity.

Now, it's a little bit difficult to show rigorously that the measure described above actually exists. A relatively easy way to do it uses an ultrafilter. The existence of the required ultrafilter is usually established by using the axiom of choice or its equivalent, Zorn's lemma. (I wrote a little bit about this here.)

In fact, any ultrafilter $\mathcal{U}$ defines a finitely additive measure by setting $P(U)=1$ if $U \in \mathcal{U}$ and $P(U) = 0$ if $U \notin \mathcal{U}$. See this for example.

On the other hand, it has been shown that any purely finitely additive measure is non-constructible: the existence of such a measure cannot be proved with the ZF axioms of set theory alone. See this paper.

Answer 2

This is a good example to show that induction can prove things "for all positive integers $n$" but that does not necessarily imply the result "for $n=\infty$."

There are still some things you can say with only the finitely additive axiom: Let $\{A_n\}_{n=1}^{\infty}$ be an infinite sequence of disjoint events. Fix $m$ as a positive integer. Since $\cup_{n=1}^{\infty} A_n \supseteq \cup_{n=1}^m A_n$ we conclude: $$P[\cup_{n=1}^{\infty} A_n] \geq P[\cup_{n=1}^m A_n] = \sum_{n=1}^m P[A_n] $$ where the final equality holds by finite additivity. Taking a limit as $m\rightarrow \infty$ gives: $$ P[\cup_{n=1}^{\infty} A_n] \geq \sum_{n=1}^{\infty} P[A_n]$$ The reverse inequality requires the countable additivity axiom (as described in the aduh answer). As a sanity check, notice that the above inequality indeed holds for the "fair integer lottery" example from aduh's answer, using $A_n = \{n\}$ for $n \in \{1, 2, 3, ...\}$ (and redefining that example only over positive integers, rather than all integers, for simplicity), since $1 \geq \sum_{n=1}^{\infty} 0 = 0$.