Correlated samples due to Metropolis algorithm

217 Views Asked by At

The Wikipedia article about the Metropolis algorith notes one disadvantage as follows:

The samples are correlated. Even though over the long term they do correctly follow P(x), a set of nearby samples will be correlated with each other and not correctly reflect the distribution.

So, can't one just randomize the array of collected samples at the end? Won't this get rid of the sample-to-sample cor-relatedness?

1

There are 1 best solutions below

2
On

It gets rid of (most of) the local correlation, but in most applications that's irrelevant. Typically one wants to take an average over all samples, which doesn't depend on the order. The variance of the sample average is increased due to the correlation, and you can't change that by reordering.

However, if your application for some reason requires uncorrelated pairs or tuples of samples, then reordering will help. Even better than randomising the order, though, would be to deliberately pair values that are far apart, e.g. if you need pairs and take $2L$ samples, pair sample $j$ with sample $j+L$.