I have an elementary question on sampling without replacement, but I thought I must ask here, to be sure I've understood it.
Example. There are 100 slips of paper in a hat, each of which has one of the numbers $1,2,3,\ldots,100$ written on it, with no number appearing more than once. Five of the slips are drawn one at a time.
Consider sampling without replacement (with all sets of five slips equally likely to be chosen).
What is the distribution(PMF) of the value of the $j$th draw?
Solution.
The author in the book writes,
Let $Y_{j}$ be the value of the $j$th draw. By symmetry, $Y_{j}$ follows $\text{DUnif}(1,2,\ldots,100)$. Here learning any $Y_{i}$ gives information about the other values (so $Y_{1},\ldots,Y_{5}$ are not independent), but symmetry still holds since, unconditionally, the $j$th slip drawn is equally likely to be any of the slips.
The author writes that the $j$th draw is essentially equivalent to sampling one object from the set $\{1,2,\ldots,100\}$. If sampling were with replacement, this is certainly true. But, when sampling without replacement, wouldn't I be left with only $100-(j-1)$ numbers to choose from, on the $j$th draw? I know there's a fallacy in my thinking there. But, I can't tell what.
Any inputs and intuition would be great help!
Cheers and thanks,
Quasar C.
Sampling without replacement would make a difference if you knew what you've already sampled, but you don't know what you've drawn before the jth draw.
It's like computing the probability of drawing ace of spades from a nonstandard deck with one unknown card already removed. It's not 1/51. It's 1/51 assuming the card removed is not the ace of spades.
So it's $(1/51)(51/52) + (0/52)(1/52) =1/52$.
In your case the probability that $Y_j$ is for example 10 is not $1/(100-(j-1))$ but rather
$1/(100-(j-1)) (100-(j-1))/(100) + 0/(100-(j-1)) (j-1)/(100) = 1/100$
Hence the $Y_j$'s are identically distributed but not independent.
So for example
$P(Y_2 = 67) = 1/100$
But
$P(Y_2 = 67 | Y_1 = 41) = 1/99$