Prove uniform distribution

Question

For any random variable $X$, there exists a $U(0,1)$ random variable $U_X$ such that $X=F_X^{-1}(U_X)$ almost surely.

Proof: In the case that $F_X$ is continuous, using $U_X=F_X(X)$ would suffice. In the general case, the statement is proven by using $U_X=F_X(X^-)+V(F_X(X)-F_X(X^-))$, where $V$ is a $U(0,1)$ random variable independent of $X$ and $F_X(x^-)$ denotes the left limit of $F_X$ for $x\in\mathbb{R}$.

How does one easily see/show that $U_X$ in the general case is a $U(0,1)$ random variable?

Answer 1

For simplicity, write $F$ for $F_X$. For each $u \in (0, 1)$, by the independence of $V$ and $X$, we have \begin{align} & P[U_X \leq u] \\ = & P[F(X-) + V(F(X) - F(X-)) \leq u] \\ = & \int_{-\infty}^\infty P[F(x-) + V(F(x) - F(x-)) \leq u]dF(x) \\ = & \sum_{x \in J}P\left[V \leq \frac{u - F(x-)}{F(x) - F(x-)}\right] \times (F(x) - F(x-)) + \int_{x \notin J}P[F(x) \leq u]dF(x) \tag{1}\\ % = & \sum_{x \in J}I_{(F_X(x-), F_X(x))}(u)(u - F_X(x-)) + \int_{x \notin J}I_{[F_X(x), 1)}(u)dF(x) \end{align} where $J$ denotes the set of jump points of $X$.

Define $x_0 = \inf\{t \in \mathbb{R}^1: F(t) \geq u\} = F^{-1}(u)$. If $x_0 \in J^c$, then for every $x \in (-\infty, x_0) \cap J$, $u - F(x-) > F(x) - F(x-)$, hence $P[V \leq (u - F(x-))/(F(x) - F(x-))] = 1$. And for every $x \in J \cap (x_0, +\infty)$, $u - F(x-) = F(x_0) - F(x-) < F(x_0) - F(x_0-) = 0$, therefore $P[V \leq (u - F(x-))/(F(x) - F(x-))] = 0$. Hence the first term of the right hand side of $(1)$ can be written as $$\sum_{x \in (-\infty, x_0) \cap J} (F(x) - F(x-)) = \int_{J \cap (-\infty, x_0)}dF(x). \tag{2}$$

To analyze the second term, notice that if $x < x_0$, then $F(x) < u$ by definition. If $x > x_0$, then $F(x) \geq u$. Thus \begin{align} & \int_{J^c} P[F(x) \leq u] dF(x) \\ = & \int_{J^c \cap (-\infty, x_0]} dF(x) \\ = & P[X \leq x_0] - \int_{J \cap (-\infty, x_0]} dF(x) \tag{3} \end{align}

Adding $(1)$ and $(2)$ over gives $P[U_X \leq u] = u$.

If there exists $x_0 \in J$ such that $u \in (F(x_0-), F(x_0)]$, similar arguments shows that the first term of the right hand side of $(1)$ equals to $$\sum_{x \in (-\infty, x_0) \cap J}(F(x) - F(x-)) + (u - F(x_0-)) = \int_{J \cap (-\infty, x_0)}dF(x) + (u - F(x_0-)), \tag{4}$$ while the second term of it equals to $$\int_{J^c \cap (-\infty, x_0)}dF(x) = P[X < x_0] - \int_{J \cap (-\infty, x_0)}dF(x) \tag{5}$$

Adding $(4)$ and $(5)$ over gives that $$P[U_X \leq u] = P[X < x_0] + u - F(x_0-) = u.$$

Answer 2

Let $u\in(0,1)$. Show that $P(U_X< u)=u$ as follows:

Define $ x^* :=\inf\{x:F(x)\ge u\} $. Then $F(x^*-)\le u\le F(x^*)$. Moreover:

1. $X<x^*\Rightarrow U_X< u$. (Reason: If $X<x^*$ then $F(X)<u$. But $U_X\le F(X)$ since $V\in[0,1]$.)

2. $X>x^*\Rightarrow U_X\ge u$. (Reason: If $X>x^*$ then $F(X-)\ge u$. But $U_X\ge F(X-)$ since $V\ge0$.)

Therefore we can write $$ P(U_X< u)\stackrel{(2)}=P(U_X< u,X\le x^*) \stackrel{(1)}=P(X<x^*)+P(U_X< u, X=x^*) = a+b. $$ Argue that $a+b=u$. There are two cases. If $F$ is continuous at $x^*$, then $b=0$ and $a=P(X\le x^*)=F(x^*)=u$. If $F$ has a discontinuity at $x^*$, then, remembering that $u$ is between $F(x^*-)$ and $F(x^*)$, we have $$b=P\left(V<{u-F(x^*-)\over F(x^*)-F(x^*-)}\right)P(X=x^*)=u-F(x^*-)=u-a. $$

[The intuition behind $U_X=F(X-)+V(F(X)-F(X-))$ is that the second term $V(F(X)-F(X-))$ is 'bridging the gaps' in the distribution of $F(X-)$. The first term $F(X-)$ is a random variable that lives on the interval $(0,1)$ but has gaps in its support corresponding to each discontinuity in $F$. The size of the gap is exactly $F(X)-F(X-)$, and multiplying this gap by $V$ picks a uniform distance across that gap; that distance is then added to $F(X-)$. This procedure results in $U_X$ taking every possible value in $(0,1)$.]

Answer 3

Comment: This theorem is frequently used in simulation. Most pseudorandom number generators return values that behave for practical purposes as if they are randomly sampled from $Unif(0,1)$. The 'inverse CDF' or 'quantile' method can be used to transform uniform random variables into random variables with other distributions.

For example, if $U \sim Unif(0,1)$ then $\sqrt{U} \sim Beta(2,1).$ This is because the CDF of $Beta(2,1)$ is $F(x) = x^2$ for $x \in (0,1)$ so the inverse CDF (also called the quantile function) of $X$ is $F^{-1}(u) = \sqrt{u}$ for $u \in (0,1).$

In the figure below, the left panel shows the histogram of 10,000 observations simulated from $U \sim Unif(0,1),$ and the right panel shows the histogram of their square roots. Histograms are plotted so that, in each histogram, each bar contains approximately 1000 random values. For example, the images of values that happened to fall in $(0, .1)$ on the left, fall into the interval $(0, \sqrt{.1}) = (0, .316)$ on the right.

In a similar way, a random sample $X_i$ from an exponential distribution with mean 1, can be simulated from a random sample of unit uniform observations $U_i$ as $X_i = -\ln(1 - U_i)$. In order for the quantile method to be of practical use, the CDF of $X$ must exist in closed form, and solving for its inverse must be tractable.

The method also works for discrete random variables. In R statistical software the quantile function of binomial called qbinom is suitably defined to permit simulation. Thus the following R code simulates and tabulates 100,000 realizations of $X \sim Binom(3, .5).$

 > u = runif(100000, 0, 1)  # generate 100,000 unit uniform observations.
 > x = qbinom(u, 3, .5)     # transform to BINOM(3, .5)
 > table(x)/100000
 x
       0       1       2       3 
 0.12430 0.37514 0.37731 0.12325