Intuition behind generating continuous random valiables

Question

If we have a random variable $X$ with cumulative distribution function $F$ that is strictly ascending and we manage to find the inverse we can generate an instance $x_1$ from a uniformly distributed random variable $Z$ by setting $x_1 = F^{-1}(z)$.
I get the idea for discrete random variables e.g. the sum of two dices where we generate, well an actual observable sum of dots.
But with continuous random variables we've learned that any concrete value e.g. it rained exactly 5mm has probability $0$, so we are interested in the probability of ranges e.g. it rained between 4 and 6mm.
So what is this $x_1$ we created above? Is it actually the range $[0,x_1]$ and we'd have to compute $x_2$ to get an arbitrary range?
Or is it really this one number $x_1$? If so how is this possible?

Answer 1

Usually this is all done on computer, and you can simulate a (pseudo)-random number up to some finite precision after the $0 . \cdots$ such that the number is uniform in the space being sampled from $[0,1]$. Then you apply the $F^{-1}$. What this really means is that you are inverting an INTERVAL, since you don't specify the infinite expansion after the finite precision given. Inverting intervals is a concrete concept, with non-zero probabilities, with no paradoxes, although I agree, getting a true "sample" from a continuous distribution is somewhat dubious, at least in concrete computational terms, when the probability of any given number in the range is 0, as you point out.

Answer 2

A small correction: the most general way to handle this is through the quantile function of the random variable $X$, which is defined by $Q_X(p)=\inf \{ x \in \mathbb{R} : F_X(x) \geq p \}$ for $p \in [0,1]$. Then if $U$ is uniform on $[0,1]$ then $Q_X(U)$ has the same distribution as $X$. Note that the quantile function is properly defined even when $F_X$ is not invertible, but it is the inverse of $F_X$ when $F_X$ is invertible. This means that the quantile function can be used to sample discrete variables, continuous variables, or whatever variables you want.

Now why does this intuitively work? First, note that for any $u \in [0,1]$, $P(U \leq u)=u$. So now given a value of $u$ for $U$, we need to choose a value of $x$ for $X$. Ideally we would choose $x$ such that $P(X \leq x)=u$. This would work out because $u$ represents the fraction of the distribution of $X$ that $x$ should exceed. If we can do this, then we're done.

But this may be impossible, if the CDF of $X$ jumps through $u$. If this happens, we instead require $x$ to exceed at least $u$ of the distribution of $X$. This is the right thing to do because if $F_X$ jumps through $u$, then there is $x_0$ with $F_X(x_0^-)=u_-$ and $F_X(x_0)=u_+$, with $u_+>u_-$. In this case $X$ should take on the value $x_0$ with probability $u_+-u_-$. And this is exactly achieved by the infimum operation in the definition of $Q_X$, since $U \in [u_-,u_+]$ with probability $u_+-u_-$.

This whole thing is probably easier to see if you draw a picture of a CDF (with some continuous increasing regions, some flat regions, and some jumps) and look at the horizontal line $y=u$.

Another way to try to help (which is very important for computing applications) is to consider approximating $X$ by $Q_X(D)$, where $D$ is uniform on $\left \{ \frac{1}{N},\frac{2}{N},\dots,\frac{N-1}{N} \right \}$ for some very large number $N$. This amounts to approximating $X$ by a new random variable which takes on only $N-1$ values. (As an aside, for bounded variables, you can safely include $0$ and $1$ in the list of possible values of $D$, but for unbounded variables this may result in definition issues, like needing to evaluate $\ln(0)$.)

Answer 3

Realities of Discrete vs. Continuous.

Continuous random variables are a useful abstraction. However, in real life or in a computer simulation, one cannot really sample from a continuous distribution. Results need to be rounded to some number of decimal places for practical purposes. Then the results represent intervals. For example, the 'observation' 93.42 from Norm(100, 15) has to be understood as observation from the interval 93.42 $\pm$ .005.

Digital computers do not deal with the real numbers, nor even with the rational numbers. They deal with a finite 'grid' of carefully spaced rational numbers, usually extensive enough for the practical problems at hand. But one must always be wary of 'overflow' (values that exceed the finite limits) or 'underflow' (nonzero numbers so small they cannot be distinguished from 0 with any number in the grid).

When we speak of the number of heads on 10 tosses of a fair coin, we have a discrete (binomial) distribution. If there are 1000 tosses, we may continue talking about 1001 possible numbers of heads, or we may find in more convenient to talk in therms of a normal approximation.

When we speak of the heights of women in a certain sample, they are usually rounded to the nearest inch, centimeter, or (in very careful anthropometric studies and with some stretch of the imagination about measurement accuracy) perhaps millimeter. But there are so many possible rounded values that it is usually more convenient to ignore the discreteness and pretend that the values described by a normal (or other continuous) distribution. (There is probably no good purpose served by trying to figure out the probability a woman in the sample will have a height rounded to 1653mm.)

Visualizing an inverse CDF (quantile) transformation.

Suppose we want to simulate a sample of size three from $Exp(1)$ with CDF $F(x) = 1 - e^{-x},$ for $x > 0.$ Finding, the inverse of the CDF, we can simulate these values as $-\ln(1-U),$ where $U \sim Unif(0,1).$ In a particular case, the three uniform values are
$U_1 = 0.02369539,\, U_2= 0.84906618,\, U_3 = 0.24223515.$ and the corresponding three exponential values are $X_1 =0.02398064,\, X_2= 1.89091379,\, X_3 = 0.27738216,$ to as many decimal places as the display in R is programmed to show. (Note: Because $1 - U$ is also $Unif(0,1)$, this is often simplified to $X = -\ln(U)$ but that is not convenient for our demo.)

The figure below, shows how uniform values between 0 and 1 on the vertical axis are transformed to exponential values between 0 and $\infty$ on the horizontal axis. The curve is the CDF of Exp(1). Notice that the difference between two values of $U$ near 1 might show a very large difference in values of $X$. At some scale, large $X$ values might even be noticeably discrete.