A simple question that I don't manage to solve:
I can use different methods to measure a magnitude $x$. The results of these methods are correlated and have some uncertainties. Combining the results (and the uncertainties) I obtain $x$ with its uncertainty $\sigma_{x}$. Is it possible that including more methods would increase $\sigma_{x}$?
OK, lets see. It should be enough to consider the case with two measurements, $X$ and $Y$, both unbiased measurements for the quantity $\mu$, so $\DeclareMathOperator{\E}{E} \E X = \E Y = \mu$. Assume they have variances $\sigma^2_X, \sigma^2_Y$ and covariance $\sigma_{XY} =\rho \sigma_X \sigma_Y$ where $\rho$ is the correlation between them. Then use some weighted average to estimate $\mu$: $$ \hat{\mu} = w_1 X + w_2 Y $$ where $w_1 + w_2 =1 $ to get an unbiased estimator. Now calculate the variance of $\hat{\mu}$: $$ \DeclareMathOperator{\var}{Var} \DeclareMathOperator{\cov}{Cov} \begin{align} \var(\hat{\mu}) &= w_1^2 \var X + w_2^2 \var Y + 2 w_1 w_2 \cov(X,Y) \\ &= w_1^2 \sigma_X^2 + w_2^2 \sigma_Y^2 + 2 w_1 w_2\rho \sigma_X \sigma_Y \\ &= (w_1 \sigma_1 + w_2 \sigma_2)^2 + (\rho -1) 2 w_1 w_2 \sigma_X \sigma_Y \end{align} $$ This is increasing in $\rho$, so the worst case is if $\rho=1$. Then we have some cases: first, if $\sigma_X=\sigma_Y$. Then the above expression (with $\rho=1$) becomes simply $\sigma_X^2$, so the extra observation does not help, neither does is harm. But if $\sigma_Y > \sigma_X$, it will harm: Then we get , with $\sigma_Y = \alpha \sigma_X$ and $\alpha > 1$: Above expression becomes $\sigma_X^2 (w_1 + \alpha w_2)^2 > \sigma_X^2$. You can also make examples of this with $\rho$ close to 1 and $\sigma_Y > \sigma_X$. On the other hand, when $\rho < 0 $ it can never happen.