If I have a collection of numbers, I can obtain a measure of how much they're "spread" by computing the sample variance of them, i.e. $$\frac{1}{n}\sum_{i=1}^n(x_i-\mu)^2~,$$ where $\mu$ is the sample mean.
I was looking for a similar measure of "spreadness" but of a collection of probability distributions. For example, the following collection of probability distributions:
would have high variance, whereas the collection:
would have lower variance.
Is there such a generalization of this concept for a collection of probability distributions?
My thoughts so far:
It seems intuitive to me to try drawing a parallel between this and the definition of variance for real numbers, as described above.
One thing I could come up with was to define it as the sum of the squared "distances" to the "mean distribution". In the context of distributions, I would interpret the mean distribution as simply the mean of the pdfs (that is $\frac{1}{n}\sum_{i=1}^nF_i(x)$, where each $F_i$ is one of the distributions in the collection), and perhaps the "distance" as the Wasserstein metric.
That is, the variance of a collection of distributions $F_i(x)$ would be computed as
$$ \frac{1}{n}\sum_{i=1}^n\text{W}(F_i,\bar{F})^2~,$$
where $\text{W}(a,b)$ is the Wasserstein distance between distributions $a$ and $b$, and $\bar{F}$ is computed as $$\frac{1}{n}\sum_{i=1}^nF_i(x)~.$$
This doesn't seem so crazy, but it's based purely in my intuition. It would be nice to know if someone else already has a more developed notion on how to generalize variance in this direction.