Why do variances add when summing independent random variables?

518 Views Asked by At

I understand intuitively why spread would be additive, but not precisely why variances add rather than, say, the standard deviations.

And how does this relate to the pythagorean theorem/euclidean distance? It appears the SDs can be treated as vectors perpendicular to (i.e. independent from) each other and the length of the vector when you sum them is equal to the SD when you sum the distributions. (This seems similar to how the SD itself can be seen as the euclidean distance of summed perpendicular deviations, divided by $\sqrt{n}$).

(To be clear, I am not asking why variances can be added but not SDs, I am asking why variances are added rather than SDs (or anything else), so this is not a duplicate question)

Thanks so much!

Edit: I've gotten a great simple algebraic answer to my question, which is probably as far as you can go, but if anyone has some insight into the intuition behind it, that would be greatly appreciated too. It's probably like the pythagorean theorem and doesn't have a satisfying intuition, but I'd be more than happy with an explanation for why the standard deviations act like the sides of a right-angled triangle!

1

There are 1 best solutions below

1
On

I am asking why variances are added rather than SDs (or anything else)

The simply truth of this question is that variances (unlike standard deviations) are the only measure of "dispersion" that adds when adding random variables. Suppose we are given independent random variables $X$ and $Y$. Then by the definition of variance $$ \begin{aligned} \mathsf{Var}(X+Y) &=\mathsf E(X-\mu_X+Y-\mu_Y)^2\\ &=\mathsf E(X-\mu_X)^2+2\mathsf E(X-\mu_X)(Y-\mu_Y)+\mathsf E(Y-\mu_Y)^2. \end{aligned} $$ By the definition of variance (and covariance) we then get $$ \mathsf{Var}(X+Y)=\mathsf{Var}(X)+\underbrace{2\mathsf{Cov}(X,Y)}_{=0}+\mathsf{Var}(Y)=\mathsf{Var}(X)+\mathsf{Var}(Y). $$ Taking the square root of both sides gives $$ \mathsf{Std}(X+Y)=\sqrt{\mathsf{Var}(X)+\mathsf{Var}(Y)}\neq \mathsf{Std}(X)+\mathsf{Std}(Y). $$

So the fact that we add variances (and not standard deviations or any other measure of dispersion) is simply a consequence of the definitions of such objects.