Using importance sampling to simulate the mean of a normal distribution truncated to interval [0,1]

Question

So in these notes it says that importance sampling is: $$\int_F sf(s)ds = \int_G s \frac{f(s)}{g(s)}g(s)ds$$

And then it proceeds to give the following example:

---

---

In this example, if we draw from $f(x)$, are we effectively drawing from the truncated standard normal distribution? Also, can someone explain why it says that $g(x)=1$ if we draw $x$ from $U[0,1]$?

Answer 1

This is a 'toy' example because it is easier and better to do numerical integration, roughly as used to make printed normal tables, to get the correct answer. However, it is a nice simple example to get you acquainted with acceptance sampling.

Numerical integration (no simulation). So at the start, let's find the correct answer to your problem with numerical integration. In R statistical software, this can be done as follows:

integrand = function(x){x*dnorm(x)/diff(pnorm(c(0,1)))}
# 'dnorm' is std normal PDF;  'pnorm' is std normal CDF
integrate(integrand, 0, 1)
0.4598622 with absolute error < 5.1e-15

My 'function' is $xK\varphi(x),$ where $1/K = \int_0^1 \varphi(x)\,dx$ and $\varphi$ denotes the standard normal density. So for $X$ distributed according to your truncated normal distribution with denisty $\varphi^*(x),$ for $x \in (0,1)$ and $0$ otherwise, we have $E(X) = \int_0^1 x\varphi^*(s)\,dx \approx 0.4599.$ [Reality check: A sketch should convince you that the answer must be in $(0,1)$ and slightly below $1/2.$]

Brute force simulation. In R, the 'brute force' simulation method you mention amounts to the following:

x = rnorm(10^6)          #'rnorm' samples from std normal
mean(x[x > 0 & x < 1])
## 0.4598828

This is indeed an inefficient method because we are averaging over only fewer than 341,000 out of the one million sampled values of x. (This inefficiency is to be anticipated, because $1/K = P(0 < Z < 1) = .3413,$ where $Z \sim \mathsf{Norm}(0,1).$ ) I got a good answer because I used a million iterations, which would have been an unthinkable extravagance only a few years ago.

sum(x > 0 & x < 1)
## 340532

Importance sampling. In answer to one of your questions: $g(x) = 1,$ for $x \in (0,1)$ because that is the PDF of $\textsf{Unif}(0,1).$

Now, consider the equation $$\int_0^1 xf(x)\,dx = \int_0^1 x\frac{f(x)}{g(x)}g(x)\,dx = \int_0^1 xw(x)g(x)\,dx = \int_0^1 xw(x)\,dx,$$ where $w(x) = f(x)/g(x).$ Here $f(x) = \varphi^*(x)$ above. R code for the desired mean below uses all one million values sampled from $\mathsf{Unif}(0,1).$

K = 1/diff(pnorm(c(0,1)))  
u = runif(10^6)
K*mean(u*dnorm(u))
## 0.4599921

Answer 2

Importance sampling means: to compute an average of $h$ against the pdf $f$, draw samples from the pdf $g$, and average $\frac{h f}{g}$. The goal is to choose $g$ which is nearly directly proportional to $hf$, so that $\frac{hf}{g}$ is nearly constant (and therefore has small variance). On the other hand, if $hf$ has one sign and you could draw directly from $g=C hf$, then you would already know what the integral of $hf$ was: it would just be $1/C$. So you would already be done. So there is a compromise to be made.

In the problem you are looking at, if you were drawing from $f$, then yes you would be drawing from the truncated normal distribution. When you are drawing from $g$, the pdf of $U[0,1]$, that pdf is just $1$ on $(0,1)$ and zero elsewhere, by the definition of the uniform distribution.