Does the posterior approach the form of a conjugate prior in any meaningful sense?

68 Views Asked by At

The Bernstein von-Mises theorem says that, "the posterior distribution for unknown quantities in any problem is effectively asymptotically independent of the prior distribution (assuming it obeys Cromwell's rule) as the data sample grows large." Conjugate priors have the nice property that they have a closed form under Bayesian update (assuming there is only one form for the likelihood) in the sense that it remaps the parameters of the prior without changing the form.

Now, if the posterior is independent of the prior for all Cromwell's rule priors (I call them "non-hubristic"), then they should all, in some sense converge to the same form as the posterior produced by a conjugate prior. Is that the case? If so, does the convergence depend on how "distance" between functions/posteriors is defined, or will it hold for any metric/semi-metric/divergence? If not, what are the limitations on the "independence" of the posterior from the prior?

Note that I leave it as "a conjugate prior" on purpose to allow for the possibility that the posterior may be closest to a posterior from a conjugate prior with a particular set of parameters, and that the difference may be significant.