The wikipedia article on Metropolis-Hastings algorithm suggests that using all of the sample points produced by the algorithm as i.i.d samples from the underlying distribution is bad, since consecutive samples are autocorrelated. The article proposes undesirable solutions, such as only keeping every n-th sample, or using a large variance sampling function that would lead to lots of rejections. What I'm wondering is whether, given sufficiently many samples from the distribution, it is possible to randomly shuffle the samples to destroy the autocorrelation. Would the resulting points asymptotically match the desired distribution?
Edit: Reason for starting the bounty: I would like to know the answer to the following two questions:
- If one obtains a very long sequence of points from MH algorithm with some reasonable generic (e.g. gaussian) sampling function, and then permutes the points, is the resulting set of points an i.i.d sample from the underlying distribution
- Can this procedure be used in practice with finite number of points? Namely, is there a result that would bound some measure of convergence of the sampled distribution to the true distribution given the sampling function and the number of samples
It is not necessary to explicitly provide the proof. It would suffice to get a link and a conclusive answer.
If I understand your question correctly, the answer is no. The point is that many samples have almost the same value, shuffling won't change that. The autocorrelation is because they are generated sequentially, so it is not really the autocorrelation per se that is the issue, but that the error in convergence to the average is not proportional to $ 1\over \sqrt N$ but rather $1\over \sqrt {N\over \ L}$ with $L$ being the correlation length.