Can you combine the mean and variance of into a single value?

Question

I am creating an objective function for an algorithm. I have an array that I need to return a single value from (and minimise in the optimisation). For the moment I only use the mean of the array. However, I would also like to consider the variance of that array (and minimise it as well).

Either I use the variance and the mean as independent variables to minimise (not ideal in my opinion), or combine the two values into a single one. Is there a method to combine the mean and the variance of an array into a single value? I was going to simply do: $$ \frac{Var_{norm}+Mean_{norm}}{2} $$

where $Var_{norm}$ is the variance normalised and $Mean_{norm}$ is the mean if the array normalised. I have a feeling that there must be something better out there. I have been looking but cannot find anythin.

Any ideas?

Thank you

Answer 1

I don't think there is an accepted combination of those two statistics that makes theoretical sense.

I suggest you consider a weighted average $$ t \times \text{ mean } + (1-t) \times \text{ variance} $$ and experiment with various values of $t$ between $0$ and $1$ to see what works in your particular application.

If the mean and variance have substantially different orders of magnitude the ordinary average ($t= 1/2$) will skew toward the larger of the two.

Answer 2

The units of variance are the square of the units of the mean, so it does not make sense to add them. If the units of the mean are length, the units of the variance are length$^2$, so the minimization will depend on whether you measure in centimeters or meters. You can solve this by using the standard deviation instead of the variance.

If you use $\frac {\text{mean + standard deviation}}2$ you will minimize the mean $+1$ standard deviation point. You can weight them to get other points in the distribution. Maybe you want to minimize mean $+3$ sigma with the sense that you will minimize the "worst case". You would then use $\frac {\text{mean + 3*standard deviation}}4$

Answer 3

I agree with remarks that suggest that it depends on the purpose of what you are doing.

In evolutionary biology, fitness is often defined as the expectation of the number of offspring for a trait. When there are two competing traits--e.g. because there is a gene at which there are two alleles (alternative versions of the gene)--then in a large population, over the short term, and with some other assumptions about the two distributions over numbers of offspring, it's likely that the fitter trait will increase in relative frequency compared to the less fit trait. So this definition of fitness provides a useful estimate indicating what is likely to happen or what has been happening in the population.

In Natural Selection for Variances in Offspring Numbers: A New Evolutionary Principle and in other papers, John Gillespie pointed out that variance in number of offspring for organisms with a given, heritable trait can affect which trait is likely to increase in frequency over the long term.

When there are environmental variations that affect all members of a population at the same time, geometric mean number of offspring is a better predictor than the expectation. As a simple estimate of this effect, Gillespie uses

$$\mu - \frac{\sigma^2}{2}$$

where $\mu$ is the expectation of number of offspring, and $\sigma^2$ is the variance.

When there is per-organism variation in probabilities of numbers of offspring for organisms with the same trait, Gillespie uses the estimate

$$\mu - \frac{\sigma^2}{N}$$

where $N$ is the size of the population. That is, in this situation, calculating this value for each of the competing traits provides a better estimate than the expectation $\mu$ of which trait is likely to increase in frequency relative to the other one.

The point of describing Gillespie's results is not to argue that you should use either of these functions to combine the expectation and variance, but just to show that the best way two combine the two values depends on the application.

An alternative would be to leave the expectation and variance as distinct quantities, and think of them as determining a two-dimensional landscape. You want to minimize both, I gather, so the task is to find the low points on this landscape. That's more complicated than minimizing a single-valued function of expectation and variance. There might be more local minima that are not global minima, or it may be that you end up with a saddle shape in which you can't minimize both quantities, but if you figured this out, you would have more understanding of the situation.