Kolmogorov's probability axioms

Question

Why Kolmogorov's axioms are considered such a breakthrough in probability theory? They are just 3 simple statements everyone can agree with.

When creating a system of axioms like this it's necessary the list of the axioms is complete. Suppose we forget about the 3rd Kolmogorov's axiom. Then we would have 2 axioms everyone could agree with when thinking about probability. Does it mean the 2 axioms are enough to claim this is a good axiomatic system of probability? We know it's not, because there's the 3rd axiom left out. But maybe these 3 axioms are not sufficient as well in a similar manner.

Look at Euclid's fifth axiom (parallel postulate). If we ommit fifth postulate, we get hyperbolic geometry, which is certainly no what we wanted to have. A similar question arises here - are those axioms sufficient? Are we sure we won't get any unintended results just following these 3 axioms?

Or maybe the statement that a given set of axioms agrees with our intuition of, let's say, probability must itself be treated as an axiom. We cannot prove it. Kolmogorov axioms survived so many years with no major complaints, then they are believed to match our intuition regarding what probability is accurately. But there are areas where it doesn't work (like quantum mechanics, which is well known for being weird and counter-intuitive). But why those axioms apparently do work in our 'common' and 'everyday' probability problems? Maybe we haven't discovered a case where they fail?

Quoting The Logico-Algebraic Approach to Quantum Mechanics Volume I: Historical Evolution, C.A. Hooker Editor, page 172:

It is obvious that since the Kolmogorov axioms are rooted in empirical experience, any change in the theory, if by such change one wants to extend its applications to the physical world, should spring directly from some phenomenological considerations. Anticipating our discussions in the subsequent sections one might say that the point of departure for the contemplated change in the model can be traced to the remarkable discovery that the physical systems arising in quantum physics are of such nature that one is no longer entitled to make the assumption that the associated experimental proposition constitute a Boolean sigma-algebra. As a consequence, the conventional i.e. the Kolmogorov formalism of probability theory is inadequate for a precise description of these systems. As a spectacular instance of such failure we may mention the facts that the notion of disjoint events is at a somewhat deeper level and that the identity $P(A+B)=P(A)+P(B)-P(AB)$ is not always true (the examples of Feynman are concerned with this failure among other things).

Answer 1

Isn't the reason for their success precisely the fact that the Kolmogrov axioms are

• small in number
• simple staements
• everyone can agree with?

(I repeat here the points of your statement, but doesn't your quote contradict the last of these points?)

It gets a bit problematic when we talk about completeness in this context: The intent of Euclid's axioms was to describe a single abstract object, "the" geometry of "the" plane (or "the" 3D space). We might also ask: Are the three group axioms (associativity, neutral, inverse) complete? In a sense they are not, for neither the statement $\forall x,y\colon xy=yx$ nor its negation can be proved from them. But that is because these axioms are there to describe many objects (i.e., models of the axiom system). And on the other end of the spectrum there are structures that fail to be groups (such as $\mathbb N$) and therefore do not suggests themselves to be treated with group theory methods.

Kolmogorv's axioms fall more in the second category: They are applicable to many different situations. And if $P(A\lor B)=P(A)+P(B)-P(A\land B)$ does not hold in real life, then this cannot be modelled as probability just like $\Bbb N$ is no group.

Answer 2

You seem to be tackling several issues at once. First though, some inaccuracies. You write "when creating a system of axioms like these..." I'm not sure what 'these' refers to. Then you say "it's necessary the list of axioms is complete." Do you mean by 'complete' that there is only one model of the axioms (up to isomorphism)? if so, why is that necessary for modelling probability events? You comparison with the axioms of geometry is unclear as well. If you omit the fifth, you do not automatically get hyperbolic geometry, you can also get projective geometry. To claim that any of those is not what we wanted to have is peculiar, particularly from a modern perspective. Geometry encompasses much more than just Euclidean geometry. And again, even with the fifth there is not just one (up to isomorphism) Euclidean geometry, but infinitely many (of various dimensions).

Now I will try to address the question of what is so great about Kolmogorov's axiomatisation. The mathematics of probability is fraught with difficulties, both conceptual and technical. There are endless examples of seemingly simple questions that turn out to be very complicated or have severely counter intuitive answers (The Monty Hall paradox for instance). Problems that appear identical may turn out to be significantly different just because of changes in the protocol. In short, it's not easy.

Having said that, the probability theory of finite probability spaces is quite simple, at least in the sense that it is clear how to model finite probability spaces: Given a finite set of events, the probability of a subset of events is the ratio of that subset to the entire set. Sweet. From it flows quite a lot, but only when the total set of events is finite.

Often, the set of events is infinite. For instance, modelling throwing a dart at a dartboard is often done by imagining the dart board as a disk in $\mathbb R^2$, and then a throw of a dart corresponds to a choice of a point in the disk. Of course the disk has infinitely many points. What is the probability that the dart hits a given point, say the centre of the disk? Well, assuming the dart lands randomly at a uniform distribution over all points, the only possible answer is $0$. A point is just too small. This is already counter intuitive enough and raises the question of how to model all of this. Well, this is all related to the notion of how big a set is. An innocent question with a highly complicated answer. It's not simple at all to develop the theory that answers this question - measure theory. Issues related to the axiom of choice quickly creep up. A famous theorem of Vitali shows that it is impossible (assuming the axiom of choice) to meaningfully assign a measure to each and every subset of $\mathbb R$.

Now, measure theory was not developed to provide some foundations of probability theory. Instead it arose from questions of integrability. Kolmogorov's wonderful insight was that he realised the same formalism can be used to turn the intuition of what probability theory should be (as you say, pretty obvious axioms) into actual axioms. Before measure theory and Kolmogorov's seminal contribution nobody knew how to meaningfully and accurately work with infinite probability spaces. Thanks to Kolmogorov a formalism was born. Now that is truly wonderful.

Lastly, the paragraph you quote is talking about something all together different. Quantum mechanical considerations defy many conceptually obvious properties. Among them Kolmogorov's axiomatisation of probability. In the world of quantum mechanics even probability behaves differently than what we are used to. Such is life.

Answer 3

Concerning the Quantum Mechanical Probability, maybe this Wikipedia reference is a nice read:

• Probability amplitude

Especially the sections   The laws of calculating probabilities of events   and   In the context of the double-slit experiment   are relevant. In the latter section we find the formula for addition of the complex probability amplitudes $\psi$ of two independent events, say $\psi_1$ and $\psi_2$, not resulting in the "common" probability $P$: $$ P \ne \left| \psi_1 \right|^2 + \left| \psi_2 \right|^2 $$ But in the following: $$ P = \left| \psi_1 + \psi_2 \right| = \left| \psi_1 \right|^2 + \left| \psi_2 \right|^2 + 2 \left| \psi_1 \right| \left| \psi_2 \right| \cos(\theta_1-\theta_2) $$ Here $\,\theta_{1,2}$ are the (complex) arguments of $\,\psi_{1,2}$ . The last term is crucial for describing Quantum Mechanical behavior. This is the "deeper level" as mentioned in the question. In fact, as Richard Feynman says: "it contains the only mystery" (The Feynman Lectures on Physics III section 1-1).

But Quantum Mechanics is not the only context where "probability" is different from Kolmogorov probability. There are two other issues related to this that I find bothering:

• Double Think about Numerosity
• Uniform Probability and Riemann Sum

Answer 4

Kolmogorov was both interested in axioms and how probability realizes in systems. For the latter, see this paper.

Probability is notoriously difficult to correctly axiomatize. Kolmogorov's probability was a revolution in that it laid the foundations for a theory that is not only rigorous, but very applicable. The only similar "easy" example I can think of is the notion of compact sets for proving stuff in real analysis.

Kolmogorov's axioms by themselves are nothing new. However, it was Kolmogorov's reinterpretation of probability through measure theory that was truly revolutionary. This allowed for a much broader and more rigorous foundation for probability theory. Everything from Kolmogorov's 0-1 Law, to interpreting $P(A|B)$ when $P(B)=0$, becomes natural and useful in this measure theoretic approach. A further example is Brownian motion, whose rigorous foundations are solely rooted in measure theory.

Whether or not Kolmogorov's theory works in quantum mechanics is a completely separate issue. Quantum probability is a generalization, and you can find ways of connecting it in Kolmogorov's theory here.

Answer 5

Note that as stated, the 'rule,' mentioned above, has multiple interpretations. $$(A) P(A\vee B)=P(A)+P(B)−P(AB)$$ as stated an axiom of Kolmogorov's system.

Unless you interpret $$P(AB)$$ as $P(A\cap B)$ as set theoretic intersection, which is how its, or was, traditionally done.

Where $A\cap B$ is the event $E;E \in F$.

$F$ being the Bool-ean algebra of events.

Where the event $E$ , is, denoted by a set of atomic events:

$\Omega_{i};\Omega_{i} \in \Omega$, generally if not always, mutually exclusive, where $\Omega$ is the sample space .

Or in other words, the union of singleton sets: $\{\Omega_{i}\} \in F, \{\}$ ,denoting said atomic events, which are often measurable in finite cases, and thus $\in F$, the 'Boolean or sigma algebra of measurable events'. This being is often, the union of the singleton sets (translate , dis-junction, of 'mutually exclusive 'atomic event's) , that are in the common intersection of the sets $E_{A}\,,E_{B}$ .

These are the union of singleton sets in ${\Omega_{i}}\in F$ (translate , dis-junction, of 'mutually exclusive 'atomic event's; $\Omega_{i}\in \Omega$), ' which denotes the events $A , B$ respectively.

So at the end of the day, one never consider composite events really, in basic Kol-mogorov's calculus, that are not mutually exclusive.

By which I mean they are always reducible to dis-junctions of mutually exclusive events. Whilst some events in $F$ may not be mutually exclusive, what those events consist in.

That is, the elements of the sets denoting each event individually, are generally if not always mutually exclusive, are, so there are no compositions (intersection of union )or events consisting of non mutually exclusive atomic events, or other partitions.

Unless you count $\emptyset$ .

Otherwise union of singleton sets, or atomic events, or countable partitions of them (unions) in the infinite cases, are generally always be analyzed set theoretically via as the disjoint union, and set co of mutually exclusive events, and set complementation so will $A\cap B$.

Although, the probability calculus was in a sense, extended later, by both Kol-mogorov and others, as it stands, the three axioms of probability $(1)$ , $(2)$ and$ (3)$ do not contain $$(A)$$, at least not explicitly.

Everything can be done by summing up only mutually exclusive events. Its just a quicker theoretical tool.

Perhaps the real, difference is, at not so much at the level of axioms, the nature of the probability space. However, the underlying model theory, logic, measure theory and sets, on the one hand, versus, functions, vector spaces, and inner-products, and what is meant by disjoint, and complementary and closure under unions of certain forms

See Chapter 5 of 'Foundations of Measurement Volume 1: Suppes Krantz, Luce et al for more on the distinction in quantum mechanics.

and the that quantum probability may be considered non-commutative, and may see a distinction in the logic between the events(1) and (2): $(1)P([A\cup B ]\vee [B\cup C])$ and $(2)PR(A\vee B \vee C)$ .

Where $A\cap B$ and $B \cup C=\emptyset $.

But, in the conventional system (such as kolmogorov), $P([A \vee B ]\land [B\vee C])$, is interpreted, set-theoretically as:$ P([A\cup B ]\cap [B\cup C])=P({B})$ for example.

Where $A , B C$ are the atomic, mutually exclusive and exhaustive events.

Thus, the probability calculus, really only contains $$(2)$$ interpreted in terms of set theoretic intersection, comple-mentation and unions.

Although generally intersection is not required given specification of the unit.

$A\cap B$ will generally be an event $\in \mathbb{F}\subseteq=2^{\Omega}$ , in algebra of events $F$. That is a set of mutually exclusive elementary outcomes ${D,C...}$ or a singleton set,${D}$\in $F$, $D$\in $\Omega$ an atomic event, singleton,or union of said mutually exhaustive events, generally, in the algebra of events, as will every other event, generally $A\cap B=$ ,sing set theoretic comple-mentation.

Moreover, $P(AB)$ is not interpreted as some kind of product or multiplicative event it is always either a union an atomic events, or the empty event or the unit event at the end of the day, when one using set theoretic intersection for a singular probability space.

There are no distinct product events in the algebra over and above these.

$$(1)P(A \vee B)=P(A)+P(B)−P(A\land B)$$. $$(2)P(A\cup B)=P(A)+P(B)−P(A\cap B)$$.

It has been debated whether Kolmorogov's original axiom set is incompatible with quantum mechanics.

Maybe its official extension to measure theory, to joint algebras, product sets, what have you, to definite laws of large numbers, independence and so forth. But the original axioms say nothing about.

A lot of the counter-examples I have sen in other articles, have nothing to with Kolmogorov's official formulation which says nothing about products, independence,Bayes' axiom or the multiplicative /product'axioms'.

Which are put forward as mere definitions, and by definition this means that:

$P(A|B)$ is a word that merely denotes $\frac{P(AB)}{P(B)} .

Where $P(A|B)=\frac{P(AB)}{P(B)}$ then says that the ratio of two probabilities $\frac{P(AB)}{P(B)} =\frac{P(AB)}{P(B)}$ equals the the ratio of the same two probabilities (ie nothing substantive).

This is because, conditional probabilities, are not explicitly part of the axiomatic structure derived from the measure theoretic interpretation of probability,if you can call it that.

Quite, unlike the addiv-itiy of probabilities over disjoint unions, which was 'derived' from the addivity of under-lying inter-pretation (measure).

Nor is, $P(A|B)$, the probability of a conditional even, at least in the canonical calculus of probability. Nor, are the definitions of independence and Bayes rule (definition really).

These, independence, Bayes rule, conditional probabilities etc, are not officially not axioms of Kol-mogorov's system. hey are definitions!

There may have been reasons,why Kolmorogov, did not officially publish his results earlier, which were published instead at about the same time that Quantum probability-logic of Von Neumann /Birk-hoff etc. These works, have the same official vintage.

Kol-mogorov, may have been aware of some of the issues. Perhaps, part of the reason for his his being tentative about considering or formalizing the product definition, Bayes rule, and independence, as axioms of probability.

Whilst, Kol-mogorov never did officially include these as axiom but only definitions, the probability calculus was extended later, by both himself and others to accommodate Bayes rule to put it on a more formal axiomatic like standing.

Nonetheless, as it stands, the three axioms of probability still do not contain contain it (Baye's Rule) or independence, and thus the 'product rule' ,$(2)$, or the rule of 'total probability', $(1)$ , as below, as axioms:

$$(1)P(A \\vee B)=P(A)+P(B)−P(A\land B)$$ .

$$(2)P(A\cup B)=P(A)+P(B)−P(A\cap B)$$.

The only official use of is $(2)$ , as an axiom, where roughly : $P(A\cup B)= P(A\\B)+P(B)$. But this makes no use of the notions of definitions of conditional probabilities, multip-licativity of probabilities as in the product rule, or independence.

Axioms: Where a Probability space is a triple : $$\langle \Omega,\mathbb{F}, P\rangle$$.

Where, $\Omega$ is the sample space, the set of mutually exclusive and exhaustive atomic events, ,often called single-tons events,$\in$ the algebra $\mathbb{F} $. Where, $\mathbb{F} \subseteq \mathcal{P(\Omega)}=2^{\Omega}$, is an Bool-ean algebra events, a set of measurable subsets, of $\Omega$, closed under the unit event (certain event:$\Omega$, complementation, and countable union.

$$(1)P(E)\in \mathbb{F} ,P(E)\geq 0\quad \forall E\in F$$ $$(2) P(\Omega)=1$$.

$$(3)P (⋃^{i=\infty}_{i=1}E_{i}) = $$. $$\lim_{n \to \infty} ∑^{i=n} _{i=1}P(E_{i})$$.

Where: $\forall A_{i}$ are mutually exclusive. That is, pairwise disjoint.

A countable sequence of , pairwise-disjoint sets, which requires an axiom of continuity, and sometimes is formalized as fourth axiom distinct from finite additivity.

If you read Kol-mogorov's main work, he states many tentative axioms such as axiom $4|5$ the frequency principle which he decided to reject in the end .

Whilst Kol-morogov is often considered to be a frequentist in certain circles, in the end he rejected the formal frequent-ism of Von Mises/Reichen-bach, as did Von -Neumman in Quantum- Mechanics .Kol-morogov may have been still a frequentist at heart. He said explicitly that his model did not have a great deal to say, or was neutral about the world, and that frequency data, is how you obtain information, but formally speaking he rejected it in the end.

Any countable sequence of disjoint sets:

with mutually exclusive events ,roughly: $E_{1} ,\, E_{2} \text{..}\in F$ , satisfies:

. $$P(⋃_{i=1}^{\infty}E_{i})= \, \sum_{i=1}^{\infty }P(E_{i}\,)$$.