Find the value of $\mathbb{E}(X_1+X_2+\ldots+X_N)$ of i.i.d random variables $X_i$s.

Question

Let $ X_1,X_2,X_3 ,\ldots$ be a sequence of i.i.d. random variables with mean $1$. If $N$ is a geometric random variable with the probability mass function $\mathbb{P}(N=k)=\dfrac{1}{2^k}$;
$k=1,2,3,\ldots$ and it is independent of the $X_i$'s, then $\mathbb{E}(X_1+X_2+\ldots+X_N)=$?

My work:

Since mean of $X_i$s are $1$, so $\frac{X_1+X_2+\ldots+X_N}{N}=1$. Hence $\mathbb{E}(X_1+X_2+\ldots+X_N)=\mathbb{E}(N)=\sum_{k=1}^{\infty}k\mathbb{P}(N=k) = \sum_{k=1}^\infty \dfrac{k}{2^k}=2$.

Is my solution correct? Is this the right process to do it? Help please. Thanks.

Answer 1

Only a way to solve it correctly. In the answers of Ant and André it is explained what you did wrong.

If $S_N:=X_1+\cdots+X_N$ then: $$\mathbb ES_N=\sum_{k=1}^{\infty}\mathbb E(S_N\mid N=k)\Pr(N=k)$$

Observe that $\mathbb E(S_N\mid N=k)=\mathbb ES_k=\mathbb EX_1+\cdots+\mathbb EX_k=k$.

I leave the rest to you.

---

Edit

To explain why $\mathbb E(S_N\mid N=k)=\mathbb ES_k$, observe that $\left\{ S_{N}\in A\wedge N=k\right\} =\left\{ S_{k}\in A\wedge N=k\right\} $ for any measurable $A$, leading to:$$\Pr\left(S_{N}\in A|N=k\right)=\frac{\Pr\left(S_{N}\in A\wedge N=k\right)}{\Pr\left(N=k\right)}=\frac{\Pr\left(S_{k}\in A\wedge N=k\right)}{\Pr\left(N=k\right)}=\Pr\left(S_{k}\in A\right)$$

The last equation is a consequence of the fact that $N$ and $S_k$ are independent. So the distribution of $S_N$ under condition $N=k$ is the same as the distribution of $S_k$.

Answer 2

No, your process does not make any sense. Even if the mean of $X_k$ is $1$, this does not mean that $$\frac {X_1 + \dots + X_N}{N} = 1$$

for some $N$.

This is a huge misunderstanding; I'll try to briefly explain why is it so.

Random variables are not numbers

When talking about a random variable, you're talking about a function. When you write $X_1$, what you mean is $X_1(\omega)$; that is, it takes a different value for different $\omega$. As such, you cannot expect $X_1(\omega) + \dots + X_N(\omega) = N(\omega)$ to hold! The left hand side depends on the particular $\omega$, as does the right hand side. Since they are independent, this equation cannot hold for any $N$ (random variable or not)

Relationship between mean and expectation

Your confusion probably stems from the fact that if $x_i$ are numbers, then we call their "mean" (or more properly average) the number $\frac 1n\sum_{i=1}^n x_i $. Clearly you can do the same with random variables (and the result will again be a random variable, not a number) to get to

$$\frac 1n\sum_{i=1}^n X_i(\omega)$$

(note that in this context I am assuming $n$ constant, different than $N(\omega)$ random variable)

A priori, this value has no relationship whatsover with the abstract integral $E[X_1]$ (which is what we call the mean of the random variable). They are completely different things defined in completely different ways! For instance, $E[X_1]$ only depends on $X_1$ while the other is an average of many random variables.

One discovers, after some work, that if the random variables $X_i$ are independent and indentically distributed with mean $\mu \in \mathbb R$, then $$\frac 1n \sum_{i=1}^n X_i \to \mu$$

(the limit is taken a.s. and in $L^1$).

So you if you take a lot of iid random variables and you take their average, the result is still a random variable, but it's "very close" to a constant, and that constant is precisely the mean $\mu$. As $n \to \infty$, it will be "equal" to the constant $\mu$. This is called the Law of Large Numbers and it's the justification as of why we call $E[X_1]$ "mean" in the first place, but you should remember how things are defined and not get confused by words

How to solve the problem

Now, if instead of $N(\omega)$ we simply had $n$, the problem was easy; use linearity of expectation to find that

$$E(X_1 + \dots + X_n) = E(X_1) + \dots + E(X_n) = n$$

If $N(\omega)$ is a random variable, clearly you cannot do the same; you need to use the tower property of conditional expectation to write that

$$E(X_1 + \dots + X_N) = E(E(X_1 + \dots + X_n \mid N = n) ) = E(N) = \sum_{k=1}^\infty \frac k{2^k} = 2$$

Answer 3

The calculation is incompletely justified, and one should not write $\frac{X_1+\cdots+X_N}{N}=1$.

Observe that $E(X_1+\cdots+X_N\mid N=k)=k$. This is because the $X_i$ and $N$ are independent, so for fixed $k$, we are just looking at $X_1+\cdots +X_k$, and by the linearity of expectation, the expectation of the sum is $(k)(1)$.

By the Law of Total Expectation, $$E(X_1+\cdots+X_N)=\sum_1^\infty E(X_1+\cdots+X_N\mid N=k)\Pr(N=k).$$ Thus we conclude that $$E(X_1+\cdots+X_N)=\sum_1^\infty k2^{-k},$$ and you calculated this sum correctly.

Answer 4

By Wald's identity (basic version),

$$E\left[ \sum_{i=1}^{N} X_i\right] = E[N] E[X_1]$$

where

$$E[N] = \sum_{k=1}^{\infty} \frac{k}{2^k}$$

$$E[X_1] = 1$$