I want to minimize the following objective $\int_\Omega\|\nabla_x f(x;w)\|^2\,dV(x)$ against $w$ which are the parameters of the model $f$. The integral is intractable to compute in the general case and I have only found MCMC (Monte Carlo Markov Chain) effective in its estimation. Is there any efficient way to solve such a problem?
The trouble comes from the fact that in every step of the optimization, which is a gradient decent algorithm, I need to restart the Markov chain because estimating the integral using the samples generated in the previous step is not accurate due to changes in $f$. Consequently, I need to have a burn out period in every step of the optimization which makes it very inefficient. Is it possible to change the MCMC to somehow reuse/move the samples from the last step and keep the ergodicity of the chain?