What does measure zero mean in probability?

Question

If I have a continuous distribution of random variable $X$ then $P(X = k) = 0$ where $k$ is some specific value. I understand that this doesn't imply impossibility, as in the real world, no variable is completely continuous. We have limited precision for measuring physical phenomena, so while impossible from the math, it's physically possible.

But setting aside the "real-world" argument, if we're talking about probability theory in the context of measure theory, what does $P(A) = 0$ mean? If $A$ isn't impossible, what does $0$ mean, if anything? This post suggests that the $0$ assigned by the probability measure is a result of a limit, that $A$ is so unlikely that when performing a large number of trials, the frequency approaches $0$.

So then why not distinguish between $P(A) = 0$ and $P(A)$ approaches $0$ (like an infinitesimal) where the former represents $A$ being impossible, that is, $A = \emptyset$ and the latter meaning an event $A$ is possible, just extremely unlikely? Why does the probability measure assign $0$ to both?

Answer 1

This is my take on it, roughly:

The probabilities we assign have to be real numbers, so that we can make calculations with them basically. In real numbers, there is no way to distinguish between $0$ and an infinitesimal number. I am sure there are probably non-standard approaches to probability that may use infinitesimals, but this is not standard.

There is hope though. If we are to consider an outcome with $P(X=x)=0$ possible, it should mean that outcomes close to $x$ are also possible. This is exactly what is captured by a probability density function (PDF). A PDF is nonzero at $x$ if and only if the outcome $X=x$ is possible in that sense (up to some measure-theoretic nit-picking about sets of measure zero). This of course assumes that the outcomes live in some continuous space.

What then, if we had an isolated possible event? What I mean is, think for example of a dart throw. Let's say the dartboard is the unit disk $\{\Vert x\Vert\le 1\}$ in the plane. But then add a single point the disk, say $x=(2,0)$. If we throw a dart with some sort of uniform probability on this extended disk, then $P(X=(2,0)) = 0$ (where $X$ is the point we hit). Would you say it is possible to hit $(2,0)$? I don't think it makes sense to say that. I actually don't even think we could define a formal probability distribution, where $X=(2,0)$ is a possible outcome with probability $0$. On the other hand $P(X=(0,0))=0$, and this is clearly a possible event. All this corresponds to the PDF, which would have positive values on the unit disk, and $0$ elsewhere.

In short, we do distinguish between possible and impossible outcomes with PDF's. Therefore, we would not gain anything from distinguishing possible and impossible cases of $P(A)=0$.

EDIT: As Maximilian Janisch points out, we can actually (easily) make distributions like the one above. As a concrete example, let $Y\sim U([-1,1])$, and define $X=\begin{cases}Y & Y\ne0 \\ 2 & Y=0\end{cases}$. I can see that this breaks my argument a little, because now $X=2$ is possible just as much as $Y=0$ is.

In the end, I will maybe change my standpoint to this: In a continuous setting, it just isn't interesting to look at single points.

Answer 2

$\def\Fcal{\mathcal{F}}$

But setting aside the "real-world" argument, if we're talking about probability theory in the context of measure theory, what does $P(A) = 0$ mean?

In the context of measure theory, a fundamental concept in probability theory is called probability space, which consists of three elements:

• A sample space $\Omega$;
• A $\sigma$-field $\Fcal$ on the set $\Omega$;
• A probability measure $P:\Fcal\to[0,1]$.

For any element $A\in\Fcal$, which is often called an event, it is associated with a unique real number $P(A)$ in $[0,1]$. The identity $P(A)=0$ simply means "the probability measure of the event $A$ is zero".

If $A$ isn't impossible, what does $0$ mean, if anything?

You are now confusing yourself going back and forth the "real world" and the mathematical model. If you are willing to "set aside the 'real-world' argument", then you will have to define what "A isn't impossible" mean mathematically first before asking anything meaningful about it.

So then why not distinguish between $P(A)=0$ and $P(A)$ approaches $0$ (like an infinitesimal).

There is no limiting process in the quantity $P(A)$, which is a constant. What you want is that $$ P(X=k) = \lim_{n\to\infty}P(k-\frac1n \leq X\leq k+\frac1n)=0\;. $$

To gain more "mathematical" intuition, you may find this recent 3Blue1Brown video helpful: Why “probability of $0$” does not mean “impossible”.