Is this proof for inverse transform method valid for any pdf?

Question

This is from a uni course that includes a chapter about simulation, and introduces this method for random number generation.

While the theorem is about a general $f(x)$ (probability density function), it seems to me the proof assumes an uniform distribution, and therefore it's valid only for that. Am I wrong? Is this particular proof valid for any probability density function?

Theorem and proof:

Answer 1

Yes. In practice, the catch is that the inverse of the CDF is not known in a convenient form for some random variables.

Because this is a result widely used in computer simulation, maybe it will help to see how it works in practice. I will use R statistical software. Function runif samples from a uniform distribution; qexp is the inverse CDF (quantile) function of an exponential distribution.

The code below samples 100,000 observations from an exponential distribution with mean 10 (rate 0.1). For such a large sample there is pretty good fit of the histogram of the sample to the PDF of the population.

m = 10^5;  r = runif(m, 0, 1)  # vector of 100,000 from UNIF(0,1)
x = qexp(r, .1)  # random sample of 100,000 obs from EXP(rate=0,1)
hdr = "Histogram of Sample from EXP(1/10) with Density Fcn"
hist(x, prob=T, br=60, ylim=c(0,.1), col="skyblue2", main=hdr)
  curve(dexp(x, .1), add=T, lwd=2, col="red", n = 1001)

Note: For this example, you can write the CDF of the exponential distribution, find its inverse, and write explicit mathematics to match the simulation.

Addendum on sampling from a normal population (per @Henry's Comment). The CDF $\Phi(\cdot)$ of a standard normal distribution is not available in 'closed form' and so neither is its quantile function $\Phi^{-1}(\cdot).$ (Thus the crucial need for printed normal tables before the computer era.)

However, Michael Wichura (1998) has constructed rational approximations to both $\Phi(\cdot)$ and $\Phi^{-1}(\cdot),$ which are accurate to the extent expressible by double precision in software. Accordingly, R statistical software uses Wichura's approximation for its qnorm. So qnorm(runif(m)) returns a sample of size m from the standard normal distribution. [That is, qnorm(runif(m)) is equivalent to rnorm(m)--except for minor technical tweaks.]

set.seed(1234); qnorm(runif(1))  # Pseudo-random number generator...
[1] -1.207066                    #   for reproducibility
set.seed(1234); rnorm(1)         # Start with same PRNG seed...
[1] -1.207066                    #   to get same result with 'rnorm'                  


hist(qnorm(runif(10^5)), prob=T, col="skyblue2", br = 50,
    main="Standard Normal Sample")   
  curve(dnorm(x), add=T, lwd=2, col="red")

Note: For this example, you cannot write the CDF of the standard normal distribution. While the theorem in your Question remains valid and useful, this example must remain a computer demonstration.