Variance of cards without replacement

Question

So let's say that we label a deck of cards as 1-13. There is a 1-13 of hearts, diamonds, spades, and clubs. What is the variance of the sum of picking 3 cards (without replacement)?

If this was with replacement it's obviously just 3 times the variance of a single draw since variance is additive like that.

I already know that the variance of a singe draw is 14 because variance is just

$$ E[X^2]-E[X]^2 $$

For all purposes, can I just neglect the fact that they are not replaced? It makes sense that if one card was removed at random that the expected value of the next draw is the same (assuming the one removed was random and not some Baye's theorem situation where you were told which one was removed) so therefore the variance would also be treated the same way?

There was another post on this already: Variance of sum of cards with and without replacement

But I don't quite understand part of one of the solutions given: "In the case of "with replacement" () will equal $()^2$ because , will be independent and identically distributed."

How are these independent if one is picked after the other? Also wouldn't E[XY] be equal to $E[X^2]$ and not $E[X]^2$?

Answer 1

For all purposes, can I just neglect the fact that they are not replaced?

$\DeclareMathOperator{Var}{Var}$ If $X_i$ denotes the rank seen on the $i$th draw, then $E(X_1)=E(X_2)=E(X_3)$ and $\Var(X_1)=\Var(X_2)=\Var(X_3)$ when either sampling with replacement or sampling without replacement. However, you are looking to compute $\Var(X_1+X_2+X_3)$, which is affected by the mode of sampling. The reason is that the variance of a sum of random variables involves the covariance between pairs of variables. So for the purpose of computing the variance of a sum of random variables, you must note whether the cards are replaced or not; this affects the covariance.

It makes sense that if one card was removed at random that the expected value of the next draw is the same

As an aside, many people have trouble accepting that the expected value of the next draw is the same; they ask "when there is no replacement, doesn't the expected value of the second draw depend on what was drawn first?"

so therefore the variance would also be treated the same way?

Yes, the variance of an individual $X_i$ is the same on each draw; the reason is that each $X_i$ has the same distribution. However, when computing $\Var(X_1+X_2+X_3)$, we need to take account of covariance.

How are these independent if one is picked after the other?

We are in the sampling with replacement scenario, so after each pick, the drawn card is returned to the deck. So each pick is essentially drawing at random from a fresh deck. This justifies the independence assumption.

Also wouldn't $E[XY]$ be equal to $E[X^2]$ and not $E[X]^2$?

If $X$ and $Y$ are independent, then $E(XY) = E(X)E(Y)$. If also $X$ and $Y$ have the same distribution, then $E[X]=E[Y]$. Therefore $E[XY]=(E[X])^2$.

Answer 2

For a general $n \times k$ -deck (usual is $n=13, k=4$) we get the rather nice formula

$$ \text{Var}(X+Y+Z) = \frac{(n^2-1)(nk-3)}{4(nk-1)} $$

Where of course $X, Y, Z$ are three cards drawn without replacement.

This comes from calculating

$$ \mathbb{E}[(X+Y+Z)^2] - \mathbb{E}[X+Y+Z]^2 = 3\mathbb{E}[X^2] + 6\mathbb{E}[XY] - (3\mathbb{E}[X])^2 $$

Here we've used symmetry:$\mathbb{E}[Z^2] = \mathbb{E}[Y^2]=\mathbb{E}[X^2]$ and $\mathbb{E}[ZY]=\mathbb{E}[ZX]= \dots = \mathbb{E}[XY]$.

Now $\mathbb{E}[XY] = \frac{nk}{nk-1}\mathbb{E}[X]^2 - \frac{n}{nk-1}\mathbb{E}[X^2]$ like in that other question. So simplifying (and using $\mathbb{E}[X^2]= \frac{1}{n}\sum_{j=1}^n j^2 = \frac{(n+1)(2n+1)}{6}$) we arrive at the formula.