I am using an ensemble MCMC sampler which samples from the posterior distribution in a Bayesian inversion. The sampler uses $N$ "chains" which all begin from different starting points drawn from the prior distribution.
I have consistently found that some of the chains get "stuck" in a region of higher probability which results in a high autocorrelation. I've tried changing the step length, I've tried thinning, I've tried changing how I choose the starting points, etc. But no matter what I do, it always seems that at least one of the chains goes sideways.
I've also found that step length seems to be the biggest control on how many chains get stuck. If I use a large step length, then the chains are able to take a big step and get out of the local region they are stuck in. However, if I use a big step length, my rejection rate gets much larger (e.g. >90%) because most of the steps are too big and get rejected.
Regardless of the issues detailed above, the chains that do reach a region of high probability look really good. They have nice distributions around the true solution, they are uncorrelated, and they have reasonable trace plots.
Can I just exclude the "bad" chain(s)? Or is that a big no-no?
If I can't exclude the bad chain(s) then are there any suggestions on how to improve those chains?
Any info is appreciated.