When drawing a sample of size n from population of size N
This relationship holds
$$ SE\space when\space drawing\space sample\space without\space replacement\space =\space correction\space factor\space *\space SE\space when\space drawing\space with\space replacement$$
where
$$ correction\space factor = \sqrt{\frac{N-n}{N-1}} $$
N = size of population
n = size of sample
I would like to know how this relationship was derived.
Suppose you have an urn with $M$ red marbles and $N-M$ blue marbles. You can say that $N$ is the population. Then you draw $x$ red marbles and $n-x$ blue marbles, where $n$ is the sample size. Then the random variable $X$ is hypergeometric distributed. Therefore the sample has a variance of $Var(X)=n\cdot \frac{M}{N}\cdot \left(1-\frac{M}{N} \right)\cdot \frac{N-n}{N-1}$. We can replace $\frac{M}{N}$ by $p$ and get $Var(X)=n\cdot p\cdot \left(1-p \right)\cdot \frac{N-n}{N-1}$. And the variance of the sample mean is $Var\left(\frac{\sum\limits_{i=1}^n X_i}{n}\right)=Var(\overline X)=\frac{p\cdot \left(1-p \right)}{n}\cdot \frac{N-n}{N-1}$
It is obvious that $\frac{p\cdot \left(1-p \right)}{n}$ is the sample mean variance of binomial distributed variables ($\overline Y$) divided by $n$. For a large $ n$ the distribution of $\overline Y$ can approximated by the normal distribution (central limit theorem): $Var\left(\overline Y \right)\approx \frac{\sigma^2}{ n}$ The variance of $\overline X $ then is approximately $Var\left(\overline Y \right)\cdot \frac{N-n}{N-1}$. To obtain the approximated standard error of the mean we take the sqare root:
$$\sigma_{\overline x}\approx \underbrace{\frac{\sigma}{\sqrt n}}_{\text{SE with replacement} }\cdot \sqrt{\frac{N-n}{N-1}}$$