Variance of random sum of random i.i.d. variables - spot the mistake?

Question

Probably a trivial mistake, but can't seem to spot it:

Assume $X_1, \ldots, X_\tau$ and $\tau \in \{1, \ldots, n\}$ are random i.i.d variables, where $S_\tau = X_1 + \ldots + X_\tau$ denotes the random sum.

It can be shown that the following holds:

$$Var(S_\tau | \tau )=\tau Var(X_1)$$

However, from what I know,

$$Var(S_\tau | \tau) = \mathbb{E}( (S_\tau - \mathbb{E}(S_\tau|\tau))^2|\tau) = \mathbb{E}(S_\tau^2|\tau) - \tau^2\mathbb{E}(X_1)^2$$

Here $$ \mathbb{E}(S^2_\tau|\tau)=\mathbb{E}((X_1+\ldots+X_\tau)^2|\tau)=\sum_{i,j}^{n}\mathbb{E}(X_iX_j|\tau)\mathbb{1}_{\{i,j\leq \tau\}}=\mathbb{E}(X_1^2)\tau^2$$ due to independency and identical distributions.

So with my calculations I'm getting $$Var(S_\tau|\tau)=\tau^2 Var(X_1)$$

and I'm not sure where should the squared $\tau$ disappear.

I seem to be missing something, but can't spot it.

Would be grateful for any observations!

Answer 1

First of all, $\mathbb{E}[X_iX_j] = \mathbb{E}[X_i]\mathbb{E}[X_j] = (\mathbb{E}[X_1])^2 \ne \mathbb{E}[X_1^2]$

Also, there will be $\tau$ terms of $X_i^2$ and $\tau(\tau-1)$ terms of $X_iX_j$ in the expansion of the square. Therefore your second equation will become

$$\mathbb{E}(S^2_\tau|\tau)=\mathbb{E}((X_1+\ldots+X_\tau)^2|\tau)=\sum_{i,j}^{n}\mathbb{E}(X_iX_j|\tau)\mathbb{1}_{\{i,j\leq \tau\}}=\mathbb{E}(X_1^2)\tau + \tau(\tau-1)(\mathbb{E}[X_1])^2$$

$$Var(S_\tau|\tau)=\mathbb{E}(X_1^2)\tau + \tau(\tau-1)(\mathbb{E}[X_1])^2 - \tau^2\mathbb{E}(X_1)^2$$

$$= \tau\mathbb{E}(X_1^2) -\tau(\mathbb{E}[X_1])^2 $$

$$= \tau(\mathbb{E}(X_1^2)-(\mathbb{E}[X_1])^2)$$

$$=\tau Var(X_1)$$

Answer 2

I think one oopsie is: since $X_i$ and $X_j$ are independent, $\mathbb{E}[X_iX_j] = \mathbb{E}[X_i]\mathbb{E}[X_j] = \mathbb{E}[X_1]^2$ and not $\mathbb{E}[X_1^2]$! There must be another mistake, because plugging that in returns zero, but I don't see it quite yet.

There's also a much simpler proof. $var(S_\tau|\tau) = var(\sum_{i=1}^\tau X_i|\tau)$. The variance of the sum of i.i.d. random variables is the sum of the variance of any random variable in the sum, so $var(\sum_{i=1}^\tau X_i|\tau) = \sum_{i=1}^\tau var(X_1)$

Answer 3

The most succinct calculation writes the covariance of two iids using the Kronecker delta:$$\operatorname{Var}S_\tau=\sum_{1\le i,\,j\le\tau}\operatorname{Cov}(X_i,\,X_j)=\sum_{ij}\sigma^2\delta_{ij}=\tau\sigma^2.$$Your mistake, essentially, was to replace $\delta_{ij}$ with $1$ throughout.

Answer 4

Conditional on $\tau=k$, where $1\leq k \leq n$, $S_\tau=S_k=X_1+\dotsc +X_k$. For any $(X_i)$ that are IID, we have that $$Var(X_1+\dotsc+X_k)=Var(X_1)+\dotsc +Var(X_k)=kVar(X_1).$$ Thus, $$Var(X_1+\dotsc +X_\tau | \tau =k)=Var(X_1 | \tau =k)+\dotsc Var(X_n |\tau=k)$$ $$=Var(X_1)+\dotsc +Var(X_k)=k Var(X_1),$$ and finally we obtain $Var(S_\tau | \tau)= \tau Var(X_1)$.