Addition Rule for Expectations

Question

Why does the addition rule for expectations work no matter if the sub-events involved is independent or dependent?

For example:

Let $X$ be the number of aces in a 5-card poker hand. The probability that any particular card is an ace is $\frac{4}{52}$, so the expected number of aces among 5 cards dealt from a well-shuffled deck is

$E(X) = \frac{4}{52} + \frac{4}{52} + \frac{4}{52} + \frac{4}{52} + \frac{4}{52} = 5/13$

Considering the draws are without replacement, why are the probability constant here?

Reference:

Answer 1

The answer that lulu linked to is a suitable reference for why the addition rule works even without independence. But your question about why the draws have equal expectation is a good one and it shows that you have put some thought into the material.

It is certainly possible to see this in a brute-force way by separately considering the cases where the first card is an ace and the first card is not an ace, and it will work out in the end that the probability that the second card is an ace is still $\frac{4}{52}$. However, you're right to question that this doesn't seem to be how the text does this, and furthermore the brute-force method gets more complex as you go further down the list.

There is in fact, a different way to see that it will always work out to $\frac{4}{52}$ but it is a little less systematic. It boils down to seeing that there are symmetries in the distribution: this is a bit of an art but it is a really powerful technique when working with probability (often the trick is not to delude yourself into seeing symmetries that aren't really there).

For instance, instead of modeling a random hand as the result of five consecutive random draws with replacement, you could produce the same distribution by taking the top five cards from a random deck. In a well-shuffled deck, there should be no preference for how the first card behaves compared to the second card: the distribution should be the same if you switch those cards around (more formally, it is symmetric with respect to swapping any two cards.) In other words, you could justifiably imagine the second-from-top card being drawn first, without changing the probabilities! Then it is obvious that it still has $\frac{4}{52}$ chance of being an ace, just like the top one :).

A similar way to look at it is that if the probability was not $\frac{4}{52}$ then it would mean some cards are more likely than others to appear as the second card. When you put it that way, it sounds illogical because there is no reason that any given card should favor some values over others. Basically this is appealing to the fact that the distribution is also symmetric with respect to shuffling the values of the cards (e.g. switching all the hearts for diamonds).

Answer 2

Because both summation and integration are linear operators, so too is expectation.

For discrete random variables.

$\mathsf E(X+Y) ~{= \sum_x\sum_y (x+y) ~\mathsf P(X=x,Y=x) \\ = \sum_x\sum_y x~\mathsf P(X=x,Y=y)+ \sum_x\sum_y y~\mathsf P(X=x,Y=y)\\ = \sum_x x \sum_y \mathsf P(X=x,Y=y)+ \sum_y y\sum_x \mathsf P(X=x,Y=y) \\ = \sum_x x~\mathsf P(X=x)+\sum_y y~\mathsf P(Y=y) \\ = \mathsf E(X)+\mathsf E(Y) }$

For continuous RV

$\mathsf E(X+Y) ~{= \int_\Bbb R\int_\Bbb R (x+y) ~f_{X,Y}(x,y)~\mathsf d x~\mathsf d y \\ = \int_\Bbb R\int_\Bbb R x~f_{X,Y}(x,y)~\mathsf d x~\mathsf d y+ \int_\Bbb R\int_\Bbb R x~f_{X,Y}(x,y)~\mathsf d x~\mathsf d y \\ = \int_\Bbb R xf_X(x)\mathsf d x+\int_\Bbb R y f_Y(y)\mathsf d y \\ = \mathsf E(X)+\mathsf E(Y) }$

And in sigma-algera abstraction:

$\mathsf E(X+Y) ~{= \int_\Omega (X(\omega)+Y(\omega)) ~\mathsf P(\mathsf d\omega) \\ = \int_\Omega X(\omega)~\mathsf P(\mathsf d \omega)+ \int_\Omega Y(\omega)~\mathsf P(\mathsf d \omega) \\ = \mathsf E(X)+\mathsf E(Y) }$