Give a probability distribution p(x), how can one sample according to the distribution q(x) = f(x)p(x)\Z

Question

Given a probability distribution $p(x)$ with x being a continuous random variable, how can one sample according to the distribution $q(x) $$=$$ \frac{f(x)p(x)}{Z}$ where $Z$$=$$\int f(x)p(dx)$ is a normalizing constant.

Answer 1

Proposition (Inverse transform sampling). Let $F$ be a CDF and define $$ G(u) = \inf \left\{ x \in \mathbb{R} \colon F(x) \geq u \right\}. $$ Given a uniform random variable $U$ on the unit interval $(0, 1)$, $G(U)$ has CDF $F$.

In other words, if we have a uniform RNG, we can sample from any target distribution.

Proof. If $F$ has an inverse, then $G = F^{-1}$ and hence $$ \mathbb{P}(G(U) \leq x) = \mathbb{P}(F^{-1}(U) \leq x) = \mathbb{P}(U \leq F(x)) = F(x). $$

The case in which $F$ does not admit an inverse is more involved (see [1]).

In your case, $F$ is just the CDF of the random variable with PDF $q$.

[1] Walsh, John B., Knowing the odds. An introduction to probability, Graduate Studies in Mathematics 139. Providence, RI: American Mathematical Society (AMS) (ISBN 978-0-8218-8532-1/hbk). xvi, 421 p. (2012). ZBL1270.60004.