When modelling a real world event by assuming it has probability p, what are we saying/assuming about how that event behaves?

Question

There are countless books on statistics, and how to apply probability-theory to the real world. But I have never really understood what we are actually doing when we model a real world phenomenon with probability theory.

If you have real world events, and say that you model the real world, and assign probabilities to those events, what are you actually saying then? If you say that the chance that the bus will be on time have probability 0.3, what are you actually saying then? Most books I read interpret this as a long term relative frequency, that is, if you observe many "independent" such situations then the limiting frequency will go to 0.3. But this is not the definition of probability, and probability theory only says that this will happen with probability 1, not that it will happen surely.(measure 0 events etc.)

I guess what I am wondering is when we use probability in statistics and the real world, what does it mean when we say that an event have probability p. If we just are concerned with mathematics this is easy, then we are just saying that the measure of that event is p. So when we model a real world situation with probability in our abstract world, we give a real world event a measure p, but what are we actually saying about the real world then?

Answer 1

I think that the study of probability begins with our astonishing ability to imagine many different possible futures. Some of those futures are in some sense "likely" and some of those futures are "unlikely," but these concepts are rather vague and depend on our other astonishing ability to recall the events of the past. Depending on the accuracy of our memories, certain futures will "surprise" us if they occur and other future events will be met with a resigned attitude of "that's just what I expected."

This is all very vague and we want to find a better way to describe the "likelihood" of possible future events. So we begin to develop a measure of probability, whatever that is.

Some future events can be put to the test in a scientific way, with repeatable experiments. So I can, for example, roll dice and toss coins repeatedly to measure what happens. I can develop a theoretical approach to calculating probabilities for such events, using the idea of the number of possible outcomes. This leads to a belief in the measure of probability for certain simple types of events.

We then try to to extend our vocabulary to other kinds of events. This is where, in my opinion, probability theory starts to make some very extreme demands on our belief system. We are called to believe that non-repeatable events behave in the same way as repeatable events and we hope that our calculations that so far have been shown to be valid for repeatable events are also valid for discussing one-off events.

More deeply, we aren't really sure if the universe is deterministic or stochastic. If stochastic, the probability theory is probably a good model. If deterministic, then perhaps probability theory is not helpful.

The ancient Greeks had it both ways. The universe was governed by the gods (deterministic) but the gods were capricious and unpredictable, to the extent that they would decide the course of the future with the roll of dice (stochastic). This is where we get the phrase "it's in the lap of the gods" because they rolled their dice onto their laps...

Interestingly, even if the universe is deterministic, it may be so hard for us to asses all the variables required to predict the future that we are better off pretending that it is a deterministic universe after all.

Arthur C Clarke said that any sufficiently advanced technology is indistinguishable from magic. Perhaps the determinism of the gods only appears like blind chance to us...

Answer 2

This is a (good!) philosophical question not a mathematical one. In that respect it is not different from other questions related to the applicability of maths to the real world. Examples for such other questions are the physical reality of real numbers or the nature of limiting processes and infinity. And as always for those questions there will remain a certain gap which philosophy can't really close between the clearcut world of mathematics and messy reality. So do not expect final answers!

First, I suggest not to get too hung up about sets of measure zero for continuous distributions. If you are willing to accept the derivative in the concept of speed as a limit process of finite differences, you can as well focus on finite probability spaces without sets of measure zero and then take limits.

Next, you write, "Most books I read interpret this as a long term relative frequency". This is a pity since there are a few more types of interpretation. The Stanford Encyclopedia of Philosophy states in their article "Interpretations of probability" among others subjective and propensity approaches. With these you do not need an infinite amount of independent replications to postulate something.

I will try to explain those two approaches in the most simple setting, the fair coin. In the subjective approach, you argue that you believe (for whatever reasons) that one side of the coin is like the other, which is why you assign equal measure (of credibility) to the two events head and tails. In the propensity approach, you argue that the physical properties of the coin are such that both sides and hence both outcomes heads and tails are symmetric.

Subjective interpretations are great because you can assign probabilities to past events (What is the probability that the bus was on time yesterday?) or analyse deterministic computer experiments in a probabilistic fashion. The physical interpretation is nice because it creates a direct connection between maths and reality. In both approaches you end up with probabilities of "fifty-fifty" for the fair coin without requiring independent repetition of experiments.

Now, to answer your question:

"If you have real world events, and say that you model the real world, and assign probabilities to those events, what are you actually saying then?"

In the subjective approach you are actually saying nothing about the real world. You are only saying something about your subjective beliefs about the real world. In the propensity approach you postulate properties of a physical system (here: symmetry of the sides of a fair coin).

Answer 3

As several commenters have pointed out, there are multiple interpretations of probability. In my answer, I'll focus on the subjectivist interpretation.

"If you have real world events, and say that you model the real world, and assign probabilities to those events, what are you actually saying then? If you say that the chance that the bus will be on time have probability 0.3, what are you actually saying then?"

The short answer, for the subjectivist, is that you're saying something about your personal judgments. Exactly what you're saying depends on precisely what brand of subjectivism one endorses. What's interesting, and contra what g g has said, the issues here are not purely philosophical; there are mathematical results that shed light on your question. I turn to those now.

The only place to start is with De Finetti. According to De Finetti's subjectivist theory, there is some collection $\mathcal{B}$ of events of interest. An agent is to buy and sell various bets on these events. For each $B \in \mathcal{B}$, the agent's price function $p: \mathcal{B} \to \mathbb{R}$ gives the her fair price $p(B)$ for a bet that pays \$1 if $B$ occurs and nothing otherwise. A price function is said to be coherent if there is no way to buy and sell a collection of bets with our agent such that she is guaranteed a net loss. More precisely, $p$ is coherent if there do not exist $B_1,...,B_n \in \mathcal{B}$ and real numbers $d_1,...,d_n$ such that $$\sum_{i=1}^{n} d_i (\mathbf{1}_{B_i} - p(B_i)) < 0.$$ This simple set up is enough to state the very interesting

Theorem 1. Suppose that $\mathcal{B}$ is an algebra. Then $p$ is coherent if and only if it is a finitely additive probability measure.

Back to your questions: On this interpretation probabilities are coherent price functions. To say the probability that the bus is on time is 0.3 is to say that \$0.30 is your fair price for a bet that pays \$1 if the bus is on time.

Later authors working in the subjectivist tradition extended these ideas dramatically. Perhaps the most important work here is Leonard Savage's Foundations of Statistics. For Savage, the fundamental notion is not a price function, but a qualitative preference ordering over actions. For example, you might prefer taking the bus ($f$) to walking ($g$), and we write $g \precsim f$.

More precisely, Savage postulates a set of states of the world $\mathcal{S}$ and a set of outcomes $\mathcal{O}$. Actions are functions from $\mathcal{S}$ into $\mathcal{O}$. For example, suppose $s \in \mathcal{S}$ is a state in which the bus breaks down. Then the outcome $f(s)$ of taking the bus is that you're late to work. Now, as with De Finetti's notion of coherence, Savage imposes certain normative constraints, in the form of axioms, on the ordering $\precsim$. I won't list the axioms here, but, for example, $\precsim$ should be a weak order.

To get probabilities, we proceed in two steps. First it can be shown that if $\precsim$ satisfies the Savage axioms, then there exists a unique qualitative probability $\preccurlyeq$ on events in $\mathcal{B}$, defined to be an algebra of subsets of $\mathcal{S}$. The idea behind qualitative probability is the following. First, $\precsim$ induces an order on the set of outcomes $\mathcal{O}$ by identifying outcomes $o$ with constant actions $\hat o$, i.e. $\hat o(s) = o$ for all $s \in \mathcal{S}$. Continuing our example, suppose you prefer getting to work on time $o_2$ to begin late $o_1$, so $o_1 \precsim o_2$. Now consider two events, say the event $A_1$ that the bus driver is Terry Tao and the event $A_2$ that the bus driver is not a mathematician. How can we determine which event you regard as more probable given only your preferences $\precsim$? The idea is to look at two actions $f_1$ and $f_2$, say

$f_1$ returns $o_2$ if $A_1$ occurs and $o_1$ otherwise. That is, if you perform action 1, you're on time if Tao is driving and late otherwise.

$f_2$ returns $o_2$ if $A_2$ occurs and $o_1$ otherwise. That is, if you perform action 2, you're on time if the driver isn't a mathematician and late otherwise.

Suppose $f_1 \precsim f_2$. But since you prefer being on time $o_2$ to being late $o_1$, intuitively this means that you regard $A_2$ as more probable than $A_1$, i.e. $A_1 \preccurlyeq A_2$, because it's reasonable for you to prefer the action that makes your preferred outcome more probable. Using this idea to define the relation $\preccurlyeq$ we can show

Theorem 2. Let $\mathcal{B}$ be an algebra of subsets of $\mathcal{S}$. If the preference relation $\precsim$ over actions satisfies the Savage axioms, then $\preccurlyeq$ is a qualitative probability relation on $\mathcal{B}$, i.e. for $A,B,C \in \mathcal{B}$, $\preccurlyeq$ is a weak order with $\emptyset \preccurlyeq A$ and with $A \preccurlyeq B$ iff $A \cup C \preccurlyeq B \cup C$ when $A \cap C = B \cap C = \emptyset$.

The second step is to derive numerical probabilities. For this a few technical conditions are required. But assuming these technical conditions and the Savage axioms are all satisfied, we have

Theorem 3. There exists a unique, finitely additive probability measure $\mu$ on $\mathcal{B}$ that represents $\preccurlyeq$, i.e. for all $A, B \in \mathcal{B}$ we have $A \preccurlyeq B$ if and only if $\mu(A) \leq \mu(B)$.

This provides a rather remarkable answer to your question. The numerical probability 0.3 that you assign to the event that the bus is on time is a representation (in the precise sense given above) of your personal preferences. Subjective probability in the Savage framework derives from your ordinary preference for being on time as opposed to being late.

In both the De Finetti and Savage frameworks, we start with objects that are less mysterious than probability (e.g. betting behaviors or ordinary qualitative preferences) and show that probabilities can be derived from these objects. This gives precise meaning to the that claim I made at the beginning, namely that probabilities represent your personal judgments.

Answer 4

This is probably not accepted-answer material, but I'd still like to share my perspective.

So someone threw a couple of dice and you were passing by. What is the result for the sum you would bet on?

I think of Probability Theory and my fascination for it as humans being able to describe nature without having to thoroughly understand it. We make descriptions and rectify our approximations through experience. There are things we can intuitively know to have probability $1$, which is just cause-and-effect.

That's simple. When we have uncertainty, however, we try to measure it. We can't know if the bus will be late. It may. It may not. What matters is that we don't know. Without any experience whatsoever, the best bet is just saying we have a 50/50 chance. This reflects our own perspective. The driver knows if he's coming late. We don't.

However, with enough experience, we learn to adjust the probability. If out of 10 times I've taken the bus, 3 times it has been late, then I'll adjust my perspective to saying it has a probability of coming late of $.3$, with which I can make further calculations based on logic and Mathematics in general. It's a useful tool to extrapolate from finite experience.

Remember that a simple probability $p$ implies some kind of observation. The more you see, the better you can change $p$ towards either $0$ or $1$.

It's (almost) the same thing if someone throws a couple of dice and you close your eyes and if they throw the dice and you instantly see the result. The dice already have their Physics determined. The difference is what you can approximate in your head. If you close your eyes, each die has a $p=1/6$ of taking a specific value, so you know that if you wanted to bet on a number to be the sum of the dice, you should try $7$. There are more ways of having achieved a $7$ than other sums.

Would you bet on $7$? Say you had to forcefully bet on a number. Would it be $7$? Combinatorics and our description of the dice and Physics urge us to bet on $7$, but in the end you can bet for $5$ because it's just favorite number and you wanted to.

Answer 5

A very common guess about the real world is that we are observing a sequence identical and independent random variables. this is a requirement for the Law of Large Numbers and also the Central Limit Theorem.

The price of gasoline at two different time points are certainly correlated. if it is $3 per gallon this week, it's reasonable to assume it will be similar next week. The price a year from now is less predictable.

Additionally it is hard to remove correlations in a predictable way. Probability theory tells us a sequence of measurements concentrates around the mean. Which data point should be choose as a reference point for this mean? The price of gas today, last month or last year?

It might be possible to determine the mean over short periods of time. Since these fluctuations are not iid we have no way to determine the price in the future.

Still we assume the past is indicative of the future and we are often - but not always - right

Answer 6

As argued in this article, real world probabilities always arise from quantum fluctuations:

The point here is that even with all our simplifications, we have a plausibility argument that the outcome of a coin flip is truly a quantum measurement (really, a Schrödinger cat) and that the 50–50 outcome of a coin toss may in principle be derived from the quantum physics of a realistic coin toss with no reference to classical notions of how we must “quantify our ignorance”. Estimates such as this one illustrate how the quantum nature of fluctuations in the gasses and fluids around us can lead to a fundamental quantum basis for probabilities we care about in the macroscopic world.

They go on to show that this is true even when considering the nth digit of $\pi$, the argument then is that the choice for $n$ arises from the same sort of quantum fluctuations in the brain and the randomness comes from the lack of correlation between that choice and the value of the digit of $\pi$.

So, the interpretation of real world probabilities then comes down to your favorite interpretation of quantum mechanics. E.g. in the Many World's Interpretation you assume that you have identical copies in different Worlds where the different possible outcomes of the event will be realized.