Linearity in expectation/variance

32 Views Asked by At

Sort of a random question on my mind that I don't understand with respect to variance and expectation. We say that regardless of dependence, two RV $X$ and $Y$ are linear in their expectation and additive in their variance.

$$E(X - Y) = E(X) - E(Y)$$ $$Var(X - Y) = Var(X) + Var(Y)$$

But we also say that variance squares its coefficients:

$$Var(cX) = c^2Var(X)$$

Let us say $X = Y$. Now we have $Var(X + Y) = Var(X + X) = Var(2X)$

But based on these properties it seems like $Var(X + X) = Var(X) + Var(X) = 2Var(X)$ and Var(2X) = 4Var(X)$.

Now I understand this doesn't make sense in practice because if we subtract: $Var(X - X)$ will be $0$, not $2Var(X)$. However, what rule does it break? Is there just a special case where if the two RVs are identical we use an alternate eqn? Thanks!

1

There are 1 best solutions below

0
On BEST ANSWER

Your variance formula is incomplete. In general, the variance is defined as $Var(X) = Cov(X, X)$. In the following, we will use the fact that Covariance is linear in both arguments:

$$ \begin{align} Var(X+Y) &= Cov(X+Y,X+Y) \\ &= Cov(X,X)+Cov(X,Y)+Cov(Y,X)+Cov(Y,Y) \\ &= Var(X)+Var(Y)+2Cov(X,Y) \end{align} $$

So variance is only additive if $Cov(X,Y)=0$.

As an example, you can try to let $Y=X$ and this becomes $$Var(X-X) = Var(X)+Var(-X)+2Cov(X,-X)=Var(X)+Var(X)-2Var(X)=0.$$ Note: $$Var(-X)=Cov(-X,-X)=(-1)(-1)Cov(X,X) =Var(X)$$