Best measure of similarity between multivariate Gaussian distributions?

38 Views Asked by At

I am working to describe differences between a "baseline" random process $\mathbf{x}_0$ and other processes $\mathbf{x}_i$. All are discrete and approximated as multivariate Gaussian (they are generated by various climate models). An important characteristic of these processes is that their covariances have steep eigenvalue spectra, meaning that a handful of fixed covariance eigenvector patterns dominate the total process variance, while the remainder are effectively noise in the system (although where to draw that line is not always clear).

In this problem, it is well known that the means $\left<\mathbf{x}_i\right>$ differ from that of the baseline $\left<\mathbf{x}_0\right>$. I am interested in differences in their second moments, which are less well understood. As a starting point, I am testing to what extent differences in second moments are related to differences in means. Thus I would like to have some second-moment metric $d(\mathbf{x}_i,\mathbf{x}_0)$ that I could then compare to $\left<\mathbf{x}_i\right>-\left<\mathbf{x}_0\right>$.

So far, my approach to has been to use mean-removed processes $\mathbf{x}_0-\left<\mathbf{x}_0\right>$, $\mathbf{x}_i-\left<\mathbf{x}_i\right>$ to compute Kullback-Leibler and Jensen-Shannon divergences and the Mahalanobis distance. My concern is that these metrics will weight differences for large and small eigenvectors the same way, i.e. a difference in a small-amplitude noise process will contribute to metric size as much as a proportional difference in a large-amplitude, presumably physics-based process. Is there another metric (Bhattacharyya, Helliman?) that prioritizes larger eigenvalues? Possibly as simple as computing a norm on the difference $\left<\mathbf{x}_i\mathbf{x}_i^\top\right>-\left<\mathbf{x}_0\mathbf{x}_0^\top\right>$ between covariance matrices?