Show that $E(S_N ) = E(N)E(X_1)$

Question

$\{X_j\}_{j>0}$ is a sequance of independent, discrete random variable from same distribution and $N$ is independent, discrete random variable from any discrete distribution. For variable $S_N = X_1+...+X_N$ show: $$E(S_N)=E(N)E(X_1)$$ and $$Var(S_N) = E(N)Var(X_1) + Var(N)E(X_1)^2$$

So if variables are independent we can write: $$E(X_1)+E(X_2)+...+E(X_N) = E(S_N)$$ if there are E(N) variables from same distribution can we just write: $E(X_1)+E(X_2)+...+E(X_N)=E(X_1)+E(X_1)+...+E(X_1) = E(N)E(X_1)$ ? I think this is not best way to prove this exercise. I tried to solve this question with law of total expectation but with no success. For second question I have no idea. Thanks for any ideas.

Answer 1

We have by the law of total expectation that

\begin{align} \Bbb E(S_N) &= \Bbb E\Big( \Bbb E(S_N|N)\Big) \\&= \sum_n \Bbb E(S_N|N = n)\,\Bbb P(N=n) \\&= \sum_n \Bbb E\left(\sum_{i=1}^nX_i\right)\,\Bbb P(N=n) \\&= \sum_n \left(\sum_{i=1}^n\Bbb E(X_i)\right)\,\Bbb P(N=n) \\&= \sum_n n\Bbb E(X_1)\,\Bbb P(N=n) \\&= \Bbb E(X_1)\left(\sum_n n\,\Bbb P(N=n)\right) = \Bbb E(X_1)\Bbb E(N), \end{align}

where the index of summation $n$ ranges over all values of $N$.

---

Oh, there was also the second part. We can proceed in much the same manner. $\text{Var}(S_N) = \Bbb E\left({S_N}^2\right) - {\Bbb E(S_N)}^2$, so we need only calculate the first term.

\begin{align} \Bbb E\left({S_N}^2\right) &= \Bbb E\Big( \Bbb E\left({S_N}^2|N\right)\Big) \\&= \sum_n \Bbb E\left({S_N}^2|N = n\right)\,\Bbb P(N=n) \\&= \sum_n \Bbb E\left({\left(\sum_{i=1}^nX_i\right)}^2\right)\,\Bbb P(N=n) \\&= \sum_n \Bbb E\left(\sum_{i=1}^nX_i^2 + 2\sum_{1\leqslant i < j \leqslant n }X_iX_j\right)\,\Bbb P(N=n) \\&= \sum_n \left(\sum_{i=1}^n \Bbb E\left(X_i^2\right) + 2\sum_{1\leqslant i < j \leqslant n }\Bbb E(X_iX_j)\right)\,\Bbb P(N=n). \end{align}

Using the independence of the $X_i$, we have that $\Bbb E(X_iX_j) = \Bbb E(X_i)\Bbb E(X_j) = \Bbb E(X_1)^2$ and hence

\begin{align} \Bbb E\left({S_N}^2\right) &= \sum_n \left( n\Bbb E\left(X_1^2\right) + n(n-1){\Bbb E(X_1)}^2\right)\,\Bbb P(N=n) \\&= \sum_n n\left( \Bbb E\left(X_1^2\right) + (n-1){\Bbb E(X_1)}^2\right)\,\Bbb P(N=n) \\&= \sum_n n\left( \underbrace{\left[\Bbb E\left(X_1^2\right) - {\Bbb E(X_1)}^2\right]}_{\text{Var}(X_1)} + n{\Bbb E(X_1)}^2\right)\,\Bbb P(N=n) \\&= \text{Var}(X_1)\underbrace{\left(\sum_nn\Bbb P(N=n)\right)}_{\Bbb E(N)} + {\Bbb E(X_1)}^2\underbrace{\left(\sum_n n^2\,\Bbb P(N=n)\right)}_{\Bbb E(N^2)}. \end{align}

It follows that

\begin{align} \text{Var}(S_N) &= \text{Var}(X_1)\Bbb E(N) + {\Bbb E(X_1)}^2\Bbb E(N^2) - {\big(\Bbb E(X_1)\Bbb E(N)\big)}^2 \\&= \text{Var}(X_1)\Bbb E(N) + {\Bbb E(X_1)}^2\underbrace{\left(\Bbb E(N^2) - {\Bbb E(N)}^2\right)}_{\text{Var}(N)}, \end{align}

as desired.

Answer 2

The law of total expectation says that

$$ \mathbf{E}(S_N) = \sum_{n} \mathbf{E}(S_n)\mathbf{P}(N=n). $$

For the second question, the law of total variance:

$$ \operatorname{Var}(S_N) = \sum_n \operatorname{Var}(S_n)\mathbf{P}(N=n) + \sum_n\mathbf{E}(S_n)^2(1 - \mathbf{P}(N=n))\mathbf{P}(N=n) - \sum_{m \ne n} \mathbf{E}(S_m)\mathbf{P}(N=m)\mathbf{E}(S_n)\mathbf{P}(N=n) $$

The first term should give you $\operatorname{Var}(X)\mathbf{E}(N)$ and the other two terms should give you $\operatorname{Var}(N)\mathbf{E}(X)^2$