Variance of a sum of IID random variables.

Question

Let's say we have a sequence of $n$ IID random variables, $I_i$. Let's define a new random variable which is their sum:

$$S = \sum_i I_i$$

To calculate the variance of $S$ we can say -

$$V(S) = \sum_i V(I_i) = nV(I_1)$$

Or we can also say that $S$ has the same distribution as $nI_1$

So, we should have $V(S) = V(nI_1) = n^2V(I_1)$.

This is of course, in direct contradiction to what we got above. What am I missing?

Answer 1

$S$ does not have the same distribution as $nI_1.$ In fact, you have proven this by showing they have different variances. As an example, if $I_1$ can take the values $0$ and $1$ (i.e. it is a Bernoulli variable), then $2I_1$ can take the values $0$ and $2.$ However $I_1+I_2$ can take the value $0$ (if both are zero), $1$ (if one is zero and the other is one), or $2$ (if both are one).

Answer 2

We can not say $S$ has the same distribution as $nI_1$.

Example:

Toss two coins and count the heads as $I_1,I_2$ and their sum as $S$.   Now $I_1\in\{0,1\}$, as is $I_2$, and they are iid as required.   So $S\in \{0,1,2\}$ and yet $2I_1\in \{0,2\}$.   Thus we see that $S$ and $2I_1$ do not have the same support, therefore they cannot have the same distribution.

Answer 3

You're missing the difference between these two things: $$ \begin{align} & I_1 + I_2+ \cdots + I_n, \\[8pt] & I_1 + I_1+ \cdots + I_1. \end{align} $$ In one of those, the terms are independent; in the other, they're as far from independent as they can get.

NOTE: These do not both have the same distribution.