I'm trying to do a Monte Carlo approximation of an integral where the samples are not independent (how much so can be tuned by a parameter giving how often I sample). Therefore the regular expression for the standard deviation of the mean $\sigma_m\leftarrow\sigma/\sqrt{n}$ is presumably not a good error estimate.
I had the understanding that using bootstrap resampling one gets a good estimate of $\sigma_m$ regardless of whether the samples are completely independent (probably to within certain limits); for example in Newman and Barkema's Monte Carlo methods in statistical physics they write:
As we mentioned, it is not necessary for the working of the bootstrap method that all the measurements made be independent (...)
(...) make measurements at comfortably short intervals throughout the simulation so as to be sure of making at least one every correlation time or so (...). One of the nice things about the bootstrap method is that it is not necessary to compensate for this in applying the method.
However when I try to test this in my Monte Carlo it doesn't seem to work.
To test it I run the whole computation several times, gathering samples in such a way that they are not independent (but not "too dependent" either), and look at the average of the results, to get a primitive (but accurate) estimate of the standard deviation of the mean $\sigma_m$. (I have tried this with up to thousands of "means" to be sure). The number I get then is significantly bigger (often a factor of 2) than the bootstrap results of the individual means (for example taking the average of the latter).
As I change the parameter giving more independent samples the bootstrap result converges to the actual $\sigma_m$.
In fact there is usually no big difference between the bootstrap result and $\sigma/\sqrt{n}$.
I have a few ideas of what could be wrong:
- I have misunderstood; the bootstrap is not impervious to dependent samples
- The error estimate is crude enough that a factor of 2 is ok
- I'm computing the bootstrap in the wrong way
I have tried to make sure that 3 is not the case. Any suggestions?