Difference between random variable $2X$ and the sum of two independent observations of $X$?

Question

Consider this

Dice A has the numbers 1, 1, 2, 2, 2, 4. We find the expected value to be $E(X)=2$ and the $Var(X)=1$.

Dice B has the numbers 2, 2, 4, 4, 4, 8. As one would expect, $E(2X)=4$ and the $Var(X)=4$. Slightly unintuitively the variance increases by the square of the factor. However, my intuition for this, arises from the way variance is ultimately defined and calculated, we find the distance squared from the mean. Hence our factor should also be squared when determining the new variance.

Now consider this

Observations $X_1$ and $X_2$ are independent and of dice A. Combining the score of the seperate observations. We do this by drawing a table of all the possibilities, after adding the two values on the independent dices, a new probaility distribution is then constructed with the values 2, 3, 4, 5, 6, 7, 8 and there respective probailities.

As one would expect, $E(X_1+X_2)=2$, the same as $E(2X)$. However now, the $Var(X_1+X_2)=2$ and not $4$. Unfortunately, this is slightly hard to understand for me. Would we not expect the same result as $E(2X)$? Can't we treat $X_1+X_2=2X$. What is the difference between these scenarios?

Edit ( What is the intuition then, for why the variance changes by a factor equivalent to the number of independent observations, why this number and not something else? Can we use the same intuition for why the variance changes by the square of the factor for the other scenario? ( a rigorous proof also works ) )

Answer 1

$X_1+X_2$ is not like $2X$, because extreme values are less likely. For example: $$\Pr(2X=8)=\Pr(X=4)=\frac16,$$ but$$\Pr(X_1+X_2=8)=\Pr(X_1=4\text{ and }X_2=4)=\frac{1}{6}\times\frac{1}{6}=\frac{1}{36},$$ and similarly $$\Pr(2X=2)=\frac 13\text{ but }\Pr(X_1+X_2=2)=\frac19.$$ Since extreme values of $X_1+X_2$ are less likely, it makes sense that its variance is smaller.

Answer 2

Too long for a comment.

You seem to be interested in the intuition behind expectation and variance and why it behaves the way it does with sums of random variables. To understand this intuition, suffices to consider discrete/atomic random variables.

The expectation is the weighted average average of the values of the random variables. The intuition is that of average and it is well-known average is linear. Therefore $\mathbf{E}(X_1 + \lambda X_2) = \mathbf{E}(X_1) + \lambda \mathbf{E}(X_2)$ for whatever random variables $X_1, X_2$ and whatever scalar $\lambda \in \mathbf{R}.$

The variance is the expected square distance to the mean of a random variable. In symbols, $\mathbf{V}(X) = \mathbf{E}((X - \mu_X)^2),$ where $\mu_X$ is the real number $\mathbf{E}(X).$ Because of its very own definition, the variance acts quadratically on scalar multiples of $X;$ that is, if $X$ is substituted by $\lambda X$ then (since $\mu_{\lambda X} = \lambda \mu_X$ by linearity of the expectation), $\mathbf{V}(X) = \mathbf{E}((\lambda X - \lambda \mu_X)^2) = \lambda^2 \mathbf{E}((X - \mu_X)^2).$ What is interesting is how the variance behaves with the sum of two random variables. Consider $X$ and $Y$ two random variables and let $\bar X = X - \mu_X,$ $\bar Y = Y - \mu_Y.$ Then $$ \mathbf{V}(X + Y) = \mathbf{E}((\bar X + \bar Y)^2) = \mathbf{E}(\bar X^2) + 2 \mathbf{E}(\bar X \bar Y) + \mathbf{E}(\bar Y^2) = \mathbf{V}(X) + \mathbf{V}(Y) + 2\mathbf{E}(\bar X \bar Y). $$ This quantity will be less than, equal to or greater than $\mathbf{V}(X) + \mathbf{V}(Y)$ depending whether $\rho_{XY} = \mathbf{E}(\bar X \bar Y)$ is negative, zero or positive. By definition then, one says that $X$ and $Y$ are negatively correlated, uncorrelated or positively correlated depending on $\rho_{XY}.$ It can be shown, quite straightforwardly, if $X = Y$ then $\rho_{XX} = \mathbf{V}(X)$ and therefore, $\mathbf{V}(X+X) = \mathbf{V}(X)+\mathbf{V}(X)+2\mathbf{V}(X)=4\mathbf{V}(X)$ which we know should be the case. I don't think there is anything obscure or extra that needs be said here.