I have read this statement :
Trials are independent (i.e. use binomial) if sampling is done with replacement.
Trials are dependent (i.e. use hypergeometric) if sampling is done without replacement from a known population size.
Can someone explain it /proove it ?
Thanks
On
To explain the statements, consider with replacement first. If you have "with replacement", it means no matter what you do in that trial, the next trial is going to have the same starting condition as your previous trial. Hence, the probability of your outcomes is the same. This means they are independent of each other: your previous outcome has no affect on the conditions of the next trial.
If you have "without replacemnt", this means the outcome you have in one trial affects the next as the ending condition of a trial is the same as the starting condition of the next trial. Hence, the trials are dependent.
As an example, if you have one red ball and one blue ball and you are trying to take a random ball out of it. Probability is obviously $0.5$ for the first trial. Now, if you have replacement, you are guaranteed to go back to the condition that you have two balls in each trial, so probability is the same for each trial=$0.5$. However, if I don't replace the first ball that I take, then my next pick will be $100$% for the ball that is still left and $0$% for the ball already taken. Hence trials are dependent.
On
As in @Remy's example (+1), suppose you have an urn with 5 red balls and 5 black balls.
(1) You draw 4 balls with replacement and let $X$ be the number of red balls drawn. Then $X$ has a binomial distribution with $n = 4$ and $p = P(\text{Red}) = 1/2$ at each draw.
Also, $\mu =E(X) = np = 4(1/2) = 2; Var(X) = np(1-p) = 1.$ (This same distribution applies to tossing a fair coin four times and counting the number of Heads that occur.)
(2) You draw 4 balls without replacement and let $Y$ be the number of red balls drawn. Then $Y$ has a hypergeometric distribution, for which the original contents of the urn are parameters.
Again here, $\mu = E(Y) = 2,$ but as the number of balls left in the urn decreases, so do the possible choices, and the variability of the count of red balls decreases accordingly. (In particular, as the number of red balls in the urn decreases, drawing a red ball gets less likely, and chances of getting all red balls become small.) So $Var(Y) < Var(X).$ can you find the exact formula for $Var(X)?.$
The table below, made using R statistical software, shows how probabilities differ between the two distributions. (Ignore the brackets [ ].)
k = 0:4; bin.pdf = dbinom(k, 4, .5); hyp.pdf = dhyper(k, 5, 5, 4)
round(cbind(k, bin.pdf, hyp.pdf), 4)
k bin.pdf hyp.pdf
[1,] 0 0.0625 0.0238
[2,] 1 0.2500 0.2381
[3,] 2 0.3750 0.4762
[4,] 3 0.2500 0.2381
[5,] 4 0.0625 0.0238
In the figure below, heights of vertical bars show the binomial probabilities and the centers of the circles show the hypergeometric probabilities.
Can you see that hypergeometric probabilities are a little more tightly clustered around $\mu = 2?$
Consider an urn with $5$ red balls and $5$ black balls, where drawing a red ball is considered a "success".
If we replace the ball after each selection, then we always have a probability of $0.5$ of selecting a red ball next. Thus, our probability $p$ of a success stays constant.
If we do not replace the ball each time, then that affects the probability for future selections. If we initially selected a red, then we now have a probability of $\frac{4}{9}$ of selecting a red. If we selected red again, then we now have a probability of $\frac{3}{8}$ of selecting a red, and so forth.
Similarly, if we initially selected a black, then we now have a probability of $\frac{5}{9}$ of selected a red. If we selected black again, then we now have a probability of $\frac{5}{8}$ of selecting a red, and so forth.