How to sample according to the square of a density?

Question

I meet the following question in sampling of probability distributions.

Assume that we know how to sample according to a given probability distribution with the density $f$, while $f$ is not analytically known. Denote by $(x_n)$ the obtained samples. I wonder whether it is possible to use these samples $(x_n)$ to sample according to the distribution with the density $Cf^2$ where $C$ is the normalisation constant. Thank you!

Answer 1

If I understand your question correctly, you have absolutely continuous random variable $X$ having density $f_X(x)$ and wish to find a transformation $g$ such that $Y=g(X)$ has density $f_Y(y)=C(f_X(y))^2.$ Hence, if you apply $g(\cdot)$ to each observation in your sample, the transformed sample is drawn according to the desired density. Not sure how this can be done without knowing the analytic form for $f_X$ so let's assume you know it (e.g. using a parametric model).

---

Assuming $g$ is invertible, then the density of $Y=g(X)$ is given by

$$f_Y(y)=\frac{d}{dy}P(g(X)<y)=\frac{d}{dy}P(X<g^{-1}(y))=f_X(g^{-1}(y)){g^{-1}}' (y),$$

and equating to $C(f_X(y))^2$ tells us $z=g^{-1}(y)$ satisfies the differential equation

$$f_X(z)z'=C(f_X(y))^2,$$

which is solved by separation of variables:

$$z=F_X^{-1}\left(\int C(f_X(y))^2 dy\right),$$

where $F_X$ is the CDF of $X$.

---

Let's take an example: Suppose $X$ has CDF $F_X(x)=x^2$ (implying density $f_X(x)=2x$) on support $[0,1].$

Then we compute $$g^{-1}(y)=F_X^{-1}\left(\int C(f_X(y))^2 dy\right)=\sqrt {\int C4y^2 dy}=\sqrt {\frac{4C}{3}y^3+k},\\ \implies g(x)=\left(\frac{3}{4C}(x^2-k)\right)^{1/3}$$

for any constant $k$.

Answer 2

I don't think there's hope of doing this in general. In particular, there's no a priori reason to believe that $f^2$ can be normalized to a new density function at all. (Example: consider $f(x) = \frac{1}{2 \sqrt{x}}$ on $(0, 1)$.)