I am trying to understand why, in Monte Carlo importance sampling, the variance is zero when we choose a PDF that matches as closely as possible the function to integrate. The intuition is clear to me, but not the proof I am reading.
For an integral:
$$I = \displaystyle \int f(x) dx \tag{1} $$
the variance $\sigma^2$ is equal to:
$$ \displaystyle \sigma^2 = \frac{1}{N} \int (\frac{f(x)}{p(x)} - I)^2 p(x) dx \tag{2}$$
The approach is to find the minimum of the variance with the Lagrange multipliers method, considering as only boundary condition:
$$ \displaystyle \int p(x) dx = 1 \tag{3}$$
The function to minimize, as reported in the book, is:
$$\displaystyle L(p) = \int (\frac{f(x)}{p(x)})^2p(x) dx + \lambda \int p(x) dx \tag{4}$$
My first doubt is: why isn't the term $ -I$ in equation (2) considered in $L(p)$?
The partial derivative with respect to $p(x)$ of equation (4) gives:
$$p(x) = \frac{1}{\lambda}|f(x)| \tag{5}$$
My second doubt is: how can we say that, from equation (5):
$$ p(x) = \frac{|f(x)|}{\int f(x) dx} $$
Thank you!
Removing the $-I$ only changes the variance by a constant amount $-I^2/N$ (you can check the calculation simply since $I$ is constant with respect to $x$ and $\int p(x)dx=1$). So it doesn't affect the optimisation.
The values of $\lambda$ is just chosen to satisfy equation (3).