How to see these two ways of computing expectation are equivalent?

Question

Assume $$X:=\sum_{i=1}^nX_i,$$ where each $X_i$ is a Bernoulli random variable with mean $p_i$. (We do not assume $X_1,\dots, X_n$ are independent.)

Then there are two ways to compute expectation of $X$:

1. By linearilty of expectation, $EX=\sum_{i=1}^n p_i$.
2. By definition, $EX=\sum_{j=0}^nj\cdot P(X=j)$.

I am wondering is there a way to see the second equation is equal to the first one? Especially we do not assume $X_1,\dots,X_n$ are independent, so it seems even computing the second value is difficult?

Answer 1

For equivalence note that $$ X = \sum_{i=1}^n X_i = \sum_{j=0}^n j1_{\{X=j\}}$$ where $1_A$ is an indicator function that is 1 if event $A$ is true, and 0 else. Then take expectations of the above and use linearity of expectations (and $E[1_A] = P[A]$) to get $$ E[X] = \sum_{i=1}^n \underbrace{E[X_i]}_{p_i} = \sum_{j=0}^n j \underbrace{E[1_{\{X=j\}}]}_{P[X=j]} $$ Now, we cannot compute $P[X=j]$ without more information about dependence/independence. This is why linearity of expectations is so useful: It allows a simple answer of $\sum_{i=1}^np_i$ to a difficult question.

Homework:

Convince yourself the equation $\sum_{i=1}^n p_i = \sum_{j=0}^n j P[X=j]$ is true in these special cases (for which you can compute $P[X=j]$):

1) $X_i=X_1$, $p_i=p$ for all $i \in \{1, ..., n\}$.

2) $\{X_i\}$ are i.i.d., $p_i=p$ for all $i \in \{1, ..., n\}$.

Do these cases have different mass functions for $P[X=j]$? Do they still lead to the same answer for $E[X]$?

Answer 2

One approach uses the Probability-generating function of random variables. Quoting from Wikipedia, assuming $p(x) := P(X=x), $ we have

... the probability generating function of $X$ is defined as $$G(z) = E(z^X) = \sum_{x=0}^\infty p(x)z^x,$$

The relation between $G(z)$ and random variable expectation is $$ 1 = G(1), \quad E(X) = G'(1). \tag1 $$ Define the sum of random variables $$ X := \sum_{i=1}^n X_i. \tag2 $$ Then the probability generating function of $X$ is $$ G_X(z) = \prod_{i=1}^n G_{X_i}(z). \tag3 $$ Now using the Product rule from calculus, and $G_{X_i}(1) = 1$ we get $$ E(X) = G_X'(1) = \sum_{i=1}^n G_{X_i}'(1) = \sum_{i=1}^n E(X_i). \tag4 $$