Hypergeometric Random Variable Expectation

Question

In a binomial experiment we know that every trial is is independent and that the probability of success, $p$ is the same in every trial. This also means that the expected value of any individual trial is $p$. So if we have a sample of size $n$, by the linearity property of the expectation, the expected value of the same is just $n \cdot p$. This is all intuitive.

When the population size is finite and when we don't replace the items after every trial, we can't use the binomial distribution to get the probability of $k$ successes in a sample of size $n$ where the population is of size $N$ and the number of successes is $R$ simply because the probability of obtaining a success after every trial changes as the $R$ or/and $N$ change(s). So far so good. Yet when they calculate the expected value of the hypergeometric random variable, it is $(n \cdot R/N)$. This seems to me as the same as saying the probability of obtaining a success in every trial is the same ($R/N$) which is not intuitive at all because I should at least be expecting to see $N$ reducing by $1$ after every trial. I know that there's a flaw in my thinking. Can someone help point it out ?

Edit: I think I'm going to give up on understanding why the expected value of the hypergeometric random variable (HRV) is at it is. None of the answers have alleviated my confusion. I don't think I've made my confusion clear enough. My problem is I'm going about the process of finding the expected value of the HRV in the same way as that of the binomial random variable (BRV). In the BRV's case, if the sample is of size $n$ and we consider each item in the sample as random variable of its own, then $X = X_1+X_2+\cdots+X_n$. To find the $E[X]$, we simply add the $X_i$. Since an item is returned after it is checked, the probability of success does not change. In the case of the HRV, I should expect the probability of success to change because it is not returned back to the population. However, this doesn't seem to be the case. This is my problem.

Answer 1

We have $N$ balls, of which $r$ are red. We draw balls sequentially, without replacement.

Imagine the balls are distinct, they have ID numbers written on them, if you wish with invisible ink. Imagine also drawing all $N$ balls, one after the other. Then all of the sequences of ball ID's are equally likely. So the probability a particular ball is drawn $k$-th is $\frac{1}{N}$, and therefore the probability it is red is $\frac{r}{N}$.

Now let us suppose that we only draw $n$ balls. For $i=1$ to $n$, let random variable $X_i$ be $1$ if the $i$-th drawn ball is red, and $0$ otherwise. Then the number $Y$ of red balls is given by $Y=X_1+\cdots +X_n$. By the linearity of expectation, it follows that $$E(Y)=nE(X_1)=n\Pr(X_1=1)=n\cdot \frac{r}{N}.$$

Answer 2

We have a population of $N$ balls, or which $R$ are red, and extract a sample of $n$ balls (without replacement).

Suppose we line the balls in a row as we extract them.   Let $X_k$ be the indicator that the $k$-th ball drawn is red.   That means: $X_k=1$ if it is, $X_k=0$ if it is not.   It is a Bernouli random variable.

Now the expectation of this random variable is the probability that the $k$-th balls is red. $$\begin{align}\mathsf E(X_k)=&~1\cdot\mathsf P(X_k=1)\color{silver}{+0\cdot\mathsf P(X_k0)}\\ =&~ \tfrac RN \end{align}$$

The count of red balls withing the sample is $\sum_{k=1}^n X_k$.   Of course the random variables, $\{X_k\}_{k\in\{1,\,\ldots\,,n\}}$, are not independent, but here's a fun fact:

The Linearity of Expectation holds regardless of whether the random variables are independent or not.

Hence we have that:

$$\mathsf E\left(\sum_{k=1}^n X_k\right) ~=~ \sum_{k=1}^n\mathsf E(X_k) ~=~ n\cdot\frac{R}{N}$$

Answer 3

The probability of success is indeed the same on every trial.

Say you have four red marbles and six black marbles in an urn, and you sample eight of them without replacement. Suppose "success" means getting a black marble. Then the probability of success on the first trial is $6/10$. What is the probability of success on the second trial? Here's one way to look at that: \begin{align} & \Pr(\text{success on 2nd trial}) \\[6pt] = {} & \Pr\Big((\text{success on 1st trial and success on 2nd}) \text{ or } (\text{failure on 1st and success on 2nd})\Big) \\ = {} & \Pr(\text{success on 1st})\cdot\Pr(\text{success on 2nd} \mid \text{success on 1st}) \\ & {} + \Pr(\text{failure on 1st}) \cdot \Pr(\text{success on 2nd} \mid \text{failure on 1st}) \\[6pt] = {} & \frac 6 {10}\cdot \frac 5 9 + \frac 4 {10} \cdot \frac 6 9 = \frac{54}{90} = \frac{6\times 9}{10\times 9} = \frac 6 {10}. \end{align}

However, it's easier to see by symmetry: just interchange the roles of "1st" and "2nd" above, and see that no numbers change, using things like $$\Pr(\text{success on 1st trial}\mid \text{success on 2nd trial}) = \frac 5 9,$$ etc.

Answer 4

You have to rethink your intuition about the binomial RV, s.t. the intuition for the hypergeometric RV makes sense. Let's do it step by step.

CURRENT INTUITION FOR BINOMIAL: First of all let's question the intuition of the Binomial case. Let's say we have an urn with N balls, whereas R are red balls and N-R white. We want to draw red balls (successes). And we draw n times. The random variable X denotes the number of red balls we draw.

Currently your mental picture of drawing is:

1. close your eyes
2. draw (without looking)
3. open your eyes; look at the ball ("ahaa it's red/white")
4. put it back (since it's sampling with replacement)
5. repeat n-1 times

For all draws i in {1, ... n} we have: E[X_i ] = R / N. With linearity of expectation we get E[X] = n * R / N.

NEW INTUITION FOR BINOMIAL: So far so good. But now let's change your mental picture of the Binomial case. Just skip the "looking at the drawn ball":

1. close your eyes
2. draw (without looking)
3. don't open your eyes, but take a picture of the ball (e.g. with your phone) to look at it later
4. put it back (since it's sampling with replacement)
5. repeat n-1 times

Here, during the drawing process you have no information about the drawn balls. Only when you look at the pictures later, you will know. Again, E[X] = n * R / N. In the binomial case, there is no difference, if you look at the drawn balls or not. BUT in the hypergeometrical case it does make a difference. So here is the mental picture:

NEW INTUITION FOR HYPERGEOMETRIC:

1. close your eyes
2. draw (without looking)
3. keep the ball in a box (for you to look at it later; eyes still closed)
4. repeat n-1 times

So you haven't looked at the balls yet. Now, what's the probability that the first ball is a red one? That's easy, it's R / N. You have R successes and draw them from N balls uniformly.

Now, what about the probability, that the second ball (let's say you have it in your hand, eyes closed) is red (You still haven't looked at the first ball yet): Ahaa, it's R/N again. Why? Because you have N unknowns (N-2 in the urne, 1 in the box and 1 in your hand) and still R successes (most of them in the urne, and maybe one in the box and/or in your hand - you don't know where these R balls are). The same is true for the third draw, etc. Since you don't know which ones you have drawn already, the probability of a red ball in your hand does not change.

So again E[X_i] = R/N. With linearity of expectation, we get E[X] = n * R / N (which is, what we were looking for).

This is, what @André Nicolas meant in his comment of the accepted answer. There's a difference between P["second ball red"] and P["second ball red" | "first ball red"] (respectively P["second ball red" | "first ball white"]).

Hope that helps.

Answer 5

As others have pointed out, the probability of a red ball at each of your $n$ draw actually is $R/N$. They are just correlated.

You can also compute this expectation directly from the identity $$ \sum_{r=0}^n r\binom{R}{r}\binom{N-R}{n-r} = R\binom{N-1}{n-1} $$ To see this, the rhs counts the number of ways to pick a red ball and then $n-1$ other balls of any colour. The lhs counts the number of ways to pick $r$ red balls, $n-r$ white balls, and then pick a red ball. These are the same.

Since $$ N\binom{N-1}{n-1} = n\binom{N}{n} $$ by a similar argument, the expectation you want is $$ \frac{R\binom{N-1}{n-1}}{\binom{N}{n}} = n\frac{R}{N} $$