How do mathematician make sense of "outcome" and "events" in probability?

Question

One of the biggest challenge for me to understand probability is to make sense of this concept of outcomes and events. To put it plainly, it just doesn't feel like mathematics anymore when we talk about Head or Tail.

We are used to deal with sets in all other areas of mathematics e.g. the set of integers, the real line, a vector, a function, a set of sets,...Then you are hit in the face with "events" which are set of "outcomes" of an "experiment".

I cannot grasp why I feel more comfortable when people say that "$1$ is an element of the integers" than when people say "head is an element in the space of outcomes which contains head and tail".

Perhaps the latter doesn't contain any numbers? You cannot put it into a computer? Perhaps you can call "1" a number whereas head is ...an linguistic variable, a word, a string, a binary number, an "outcome"...what is head?

Perhaps the mapping is ill defined in my mind. You have "1", you define a function "+" and you take another number "1", it cranks out "2". Whereas you have "head" and you define a random variable which so happens to assign the number "1/2" to head and that is somehow a function.

Does anyone share my concerns when learning about probability? How can I let myself to see that concepts such as "events", "outcomes", "sample space", "random variable" are not so far fetched compared to other branches of mathematics?

Answer 1

The field of probability can be made mathematically rigorous. Introductory textbooks on probability tend to use terminology like 'sample space', 'outcome', 'event', and outcomes are given names like 'Heads' or 'HTT' or 'King of Spades', all in an attempt to keep things informal and intuitive. These texts often refrain from defining such concepts precisely, which leads to dissatisfaction among people like you who are mathematically oriented.

In fact all these informal terms can be couched in the familiar language of sets and functions, and you'll see this in more advanced courses. The 'sample space' is a set, typically called $\Omega$. An 'outcome' is a member of that set, typically denoted $\omega$. An 'event' is a subset of the sample space. A random variable is a function $X$ mapping $\Omega$ to the real numbers. A probability $P(\cdot)$ is a function mapping an event (i.e., a set) into the interval $[0,1]$. There are axioms that specify the properties we expect to be satisfied by this mapping. (To be rigorous you have to impose additional assumptions: on what sets are eligible to receive a probability, and what functions are allowed to be called random variables, although these conditions are unnecessary when the sample space $\Omega$ is finite. These concepts are more fully explored in a course on Real Analysis, specifically Measure Theory, which furnishes the mathematical foundation for probability theory.)

Using the language of set theory, an expression like $P(X\ge0)$ is understood as $P(\{\omega: X(\omega) \ge 0\})$, i.e., the event $\{X\ge0\}$ is the set of points in $\Omega$ where the function $X$ has value at least zero, and $P(X\ge0)$ is the value that the mapping $P(\cdot)$ assigns to this set. Eventually, though, it becomes tedious to constantly fill in the $\omega$ which is hiding in the background, and we get comfortable omitting it. (Indeed, it is possible to reason probabilistically without consciously thinking about that implicit $\omega$ underneath every event.)

Another example: If your experiment consists of tossing a coin twice, the sample space $\Omega$ has four elements, which we label $HH$, $HT$, $TH$, $TT$. We could just as well have used the less evocative names $\omega_1$, $\omega_2$, $\omega_3$, $\omega_4$ for these elements. The event "exactly one head was seen" is the event $\{HT, TH\}$ consisting of two elements (or, if you like, $\{\omega_2, \omega_3\}$). If the random variable $X$ is defined as "the number of heads seen", then the event $\{X=1\}$ is also this set, since it's the set of points $\omega$ where $X(\omega)=1$. Assuming the coin is fair, we define a mapping $P(\cdot)$ that maps each singleton set $\{\omega_i\}$ to the value $1/4$, and use the axioms of probability to deduce the value of $P(A)$ for all other sets $A$. And so on.

Answer 2

Before Set Theory (let us take ZFC as our axiomatization in this post) even could be considered as a foundation of mathematics, people had to figure out how to encode many of the objects, mathematics had already dealt with for milenia. Integers, functions, groups, graphs, ... it turned out that sets were a nice way to encode all these things in canonical ways.

Nowadays, mathematicians are so familiar with sets, that they now often deal with them in rather informal ways. Most mathematicians don't care how real numbers can be coded into sets and even if they know how to do it, they probably have different constructions in mind. But this doesn't matter, because mathematics - in large parts - is the study of structures up to isomorphisms. Returning to the previous example: Even if a mathematician has never bothered to construct "the" set of real numbers from scratch, given a precise definition he will see that there is an (informal) isomorphism between this set and his notion of reals.

The same applies to the examples, you have in mind. It really doesn't matter what "heads", "tails", ... are as sets. We may informally consider the set $\{ \text{heads}, \text{tails} \}$ and never worry about its elements. We just need a set with two distinct elements that aren't used otherwise in our scenario and assigning precise values to them can actually be harmful.

For example: Using Van Neumann's definition of natural numbers, the integer zero is coded as the empty set, so $0 = \emptyset$ and the integer one is coded as the set whose unique element is the empty set, so $1 = \{ \emptyset \} = \{ 0 \}$. Now, let's consider a function $f \colon \mathbb N \to \mathbb N$. For a subset $A \subseteq \mathbb N$, people often write (in abuse of notation) $f(A)$ for the image of $A$ under $f$. But now, $f(1)$ can be interpreted in two different ways: Either $f(1)$ means the value that $1$ is mapped to or $f(1) = f(\{0\})$ denotes the image of $\{0\}$ under $f$.

Answer 3

Don't be misled by formality, rigor, or philosophy. Probability theory is as perfectly sensible and interesting as every other topic in mathematics. Like many others (e.g., "algebra"), it is poorly named. It is about those random mechanisms (processes) that act unpredictably from action to action, called "trials," but deterministically when repeated over and over endlessly with respect to the relative frequencies of their possible outcomes per trial. For instance, a fair coin toss is random for each toss but the relative frequency obtaining a head (one of its two possible outcomes) is 1/2 when tossing endlessly.

There are different random mechanisms. There are different kinds of trials; for example we can roll a die three times. That is a trial that consists of three subtrials. Or we can roll three indistinguishable dice. Or we can roll three dice each unfair in a different way. Or we can roll a die until we get a six. Or we can roll a die until we get 1,2,1 in that order. Or we can roll a number of fair dice that depends on the previous sum of faces, beginning with one die, until the sum of the faces exceeds 15, which constitutes one trial.

The trial is part of the random mechanism.

Every trial has some number of "possible outcomes." This is a technical term.

The probabilities of the possible outcomes of each trial is also part of the random mechanism.

We are free to define some function of the possible outcomes that we call an "event." For example, for rolling a die until we get a six, there is a possibility that it will take nine or more rolls. This is not a property of the random mechanism itself. It is a function of the possible outcomes that we define ourselves and as such it is not what we are calling a "possible outcome."

"Probability theory" is about the kinds of random mechanisms including trials and probabilities of the possible outcomes, and the relationships between the possible outcomes and the events, including conditional probabilities, and unknown probabilities of the possible outcomes dependent on known probabilities of events.. This is a little world in itself within mathematics that is very interesting if you see what it's about.