Linearity of expectations when the upper limit of sum is an RV

Question

I am interested in the following situation:

Suppose $R$ is an RV taking values $r_1,\dots,r_n$ with $r_i \in \mathbb{Z}$. Then let $X = Y_1 + Y_2 + \dots + Y_R$ where each $Y_i$ is an independent identically distributed RV with expectation $\mu$. I need to compute $\mathbb{E}(X)$. The issue is that I have found two different ways of doing this that give seemingly different results:

Method I:

Observe that $$\mathbb{E}(X) = \mathbb{E}\left (\sum_{i=1}^RY_i\right)$$ Then by lineararity of expectation, $$\mathbb{E}\left (\sum_{i=1}^RY_i\right)= \sum_{i=1}^R\mathbb{E}(Y_i)=\sum_{i=1}^R\mu=R\mu$$ This doesn't make much since to me as I would expect expectation to be some fixed constant and not contain an RV.

Method II:

By the law of total expectation we have that $$\mathbb{E}(X) = \mathbb{E}(X|R=r_1)\mathbb{P}(R=r_1)+\ldots+\mathbb{E}(X|R=r_k)\mathbb{P}(R=r_k)$$ Hence $$\mathbb{E}(X) =r_1\mu\mathbb{P}(R=r_1)+\dots+r_k\mu\mathbb{P}(R=r_k)$$and therefore $$\mathbb{E}(X)=\mu\mathbb{E}(R)$$ This makes much more since to me and matches the solution in my course notes. Could someone explain why the two methods lead to different results? I'm sure I have made a logical error somewhere.

Answer 1

The first case actually gives the conditional expectation of X, given R, that is $$\mathbb E(X|R)= \mathbb E\left(\sum_{i=1}^R Y_i\,\middle|\,R\right);$$ but without going into much detail defining the conditional expectation, it is 'intuitive' that if $R$ is given, then you now that the sum has that finite and fixed number of terms ($R$), and so $$\mathbb E(X|R)= \sum_{i=1}^R\mathbb E(Y_i|R)=\sum_{i=1}^R\mathbb E( Y_i)=R\mu.$$

Two things are worth a remark:

— First, $\mathbb E(X|R)$ as any conditional expectation is itself a random variable: that is, the value of the conditional expectation depends on the (random) value that $R$ will take.

— Since $\mathbb E(X|R)$ is itself a random variable, one can take its own expectation (the regular one, unconditional), or its variance and any other moment, one can analyze its distribution... In particular it can be proven that if the expectations exist, its valid the relation $$\mathbb E\big(\mathbb E(X|Y)\big)=\mathbb E(X).$$ So there you have a way to complete or 'save' the first method.

By the way, a third method is possible: to find the exact distribution of $X$. Although this seems quite tricky, if $Y_i$s are all discrete r.v. with range included in $\mathbb N_0$, probability generating functions can be used (the PGF of $X$ it's defined as $G_X(t)=\mathbb E(t^X)$, and so is closely related to MGFs and characteristic functions). Once you know the PGF for $X$ it's easy to prove that$$\mathbb E(X)=G'_X(1^-).$$

Answer 2

The first one indeed makes no sense: the "global" expectation (without conditioning on $R$, for example) cannot be a random variable. Taking a global expectation like this must "integrate out" the randomness in the $Y_i$ and in $R$ at the same time.

The second one is called Wald's identity. It is indeed what you find provided that the $Y_i$ and $R$ are "sufficiently close" to being independent of one another. However, it is not enough to just have the $Y_i$ being independent of one another. For example, suppose you have independent Bernoulli(p) random variables $Y_i$ with $p \in (0,1)$, and $R=1-Y_1$. Then $E \left [ \sum_{i=1}^R Y_i \right ]=0$ (the sum is always either empty or consists of a single $0$) but $E[R]E[Y_1]=p(1-p)>0$.