I am currently encountering this problem:
I have an intractable distribution and I want to minimize the KL divergence of this distribution and a multivariate gaussian distribution. So we just need to find out the precision matrix.
The distribution is so complicated that it is not really possible to calculate the gradient or Hessian of the distribution function. All we can to is to evaluate the distribution function.
Is there any algorithm that actually solves such a problem or any suggestions on this? Thank you very much!
If you know the distribution function, you can generate samples from it using Markov Chain Monte Carlo (not Classical Monte Carlo). In MCMC (as it is termed) you only need the distribution function. In particular, I think the Metropolis-Hastings MCMC algorithm will work well for you. You can use this algorithm to approximate the mean vector and covariance matrix of your data. If you are not familiar with this technique, I suggest you consult with someone with experience in computational statsistics, as MCMC is much trickier to work with than standard Monte Carlo due to the autocorrelation in the generated sequence of random values.