$Var(aX) = a^2Var(X)$ but $Var(X+Y) = Var(X) + Var(Y)$. How does this make any sense?

Question

Apologies for the less than clear question, I wasn't quite sure how to phrase it.

Say you have a random variable $X ~ N(5, 10^2)$. Say you have another random variable $Y ~ N(5, 10^2)$.

Var(2X) = 4Var(X) = 4*10^2

But:

$Var(X+Y) = Var(X) + Var(Y) = 2\times 10^2$

But now a question arises: what?! X = Y so Var(2X) = Var(X+X) = Var(X+Y) and yet two different answers are reached using the two distinct accepted formulas. How does this make any sense? What's going wrong? It shouldn't matter what we call the random variables, and yet it seems as if it does.

Answer 1

Because in general

$$ \mathbb{V}{\rm ar}(X + Y) = \mathbb{V}{\rm ar}(X) + \mathbb{V}{\rm ar}(Y) + \color{red}{2\operatorname{\mathbb{C}{\rm ov}}(X,Y)} $$

in your last step you missed to include the covariance of $X$ with $X$

Answer 2

the variable $2X$ is not the same as the sum of two identically distributed variables $X_1+X_2$

Answer 3

$\newcommand{\v}{\operatorname{var}}$ $\v(X+Y) = \v(X) + \v(Y)$ if $X,Y$ are independent, or even if they're only uncorrelated.

$\v(X+X) \ne \v(X)+\v(X)$ since $X,X$ are nowhere near independent.

Similarly $\v(X+X+X)\ne\v(X)+\v(X)+\v(X),$ etc.

Answer 4

Note:

If $X$ and $Y$ are independent,$Var(X+Y)=Var(X)+Var(Y)$, as $2Cov(X,Y)=0$

Recall:

$Var(X)=E(X^2)-\left[E(X)\right]^2$

Where $E(X)=\int_{-\infty}^\infty xf(x)dx=\mu$ and $E(X^2)=\int_{-\infty}^\infty x^2 f(x)dx$

Thus, for $Var(X+Y)$, $X$~ $N(5, 10^2)$, $Y$~ $N(5,10^2)$,

where $X$ & $Y$ are independent:

$$Var(X+Y)=E[(X+Y)^2]-[E(X+Y)]^2$$

$$=E[X^2+2XY+Y^2]-[E(X)+E(Y)]^2 \space\space\space\space\space\space\textit{ (because}\space X \space \textit{and}\space Y \textit{are independent} )$$

$$=E(X^2)+2E(X)E(Y)+E(Y^2)-\left([E(X)]^2+2E(X)E(Y)+[E(Y)]^2 \right)$$

$$=\int_{-\infty}^\infty x^2 \frac{1}{\sqrt{2 \pi}10}e^{-\frac{(x-5)^2}{2(10^2)}}dx+2(5^2)+\int_{-\infty}^{\infty} y^2 \frac{1}{\sqrt{2 \pi}10}e^{-\frac{(y-5)^2}{2(10^2)}}dy-\left( 5^2+2(5^2)+5^2 \right)$$

$$=125+50+125-100=200 \space\space\space \textit{(After solving the integrals)} $$

Thus, $Var(X+Y)=Var(X)+Var(Y)=10^2+10^2=200$

$\mathbb{Q.E.D.}$

Hope this is helpful!

Answer 5

X is distributed identically and independently to $Y$, but $X$ IS NOT $Y$. Let's say $X$ and $Y$ each represent a die roll. If you roll $X$ and $Y$, you might get a $2$ and a $3$, which clearly are not equal. If you compare $X$ to $X$, both are $2$. For any pair of die rolls, the value of $X$ will always be equal to the value of $X$, but will not necessarily be equal to the value of $Y$, and the same goes for any pair of iid $X$ and $Y$.

Answer 6

let's make a very simple example.

assume that X is the result from rolling an ordinary dice, with possible values 1, 2, 3, 4, 5, and 6. the notation 2X then means that you roll your dice once and multiply the result by 2. that leads to the set 2, 4, 6, 8, 10, and 12 of possible values (with equal probabilities, if the dice is fair). now imagine rolling your dice twice (or using two dice, and roll each one once). X+X is now the sum of the two results from the two (independent) rolls. you could, for example, get a 1 from the first roll and a 4 from the second, and your X+X would be 5. this value was not among those possible in the first situation! in fact, all numbers from 2 through to 12 are possible results now, including all odd numbers (and the probabilities come in a triangular shape). in this sense, 2X and X+X are not the same random variables!

in order to avoid confusion, you would usually write X+Y or X1+X2 (and something like Xi iid) instead of X+X. (and if you carefully stick to this convention then you could even say that X+X=2X, but then be aware that both X must mean the same result of the same random experiment.)

one last remark: the fact that the distribution of X1+X2 is triangular hints to the central limit theorem which is exactly about how the sum of n variables shows less variability than just n times one single variable (because in the sum extremes can mutually cancel each other out).