$p$ is a prime only if
$\forall n \in\{ 2, 3, .. ,\lfloor \frac{p}{2}-1\rfloor, \lfloor \frac{p}{2}\rfloor \}$: $\dfrac{(p-1)!}{n!(p-n)!}\in \mathbb N$
The remainders and $n$'s that don't divide when $p$ is not prime seem to be factors or contain factors.
What is the proof or how does this work?
(This was re-tested and works.)
Quite an old question, still unanswered here (surprisingly), with a ready answer on Wikipedia.
If $p$ is prime, then $p$ divides $\binom{p}{n}$ for any $0<n<p$ (since $p$ divides the numerator, but not the denominator, of $\binom{p}{n}=\frac{p(p-1)\cdots(p-n+1)}{n!}$). Conversely, if $p$ is composite, and $q$ is a prime divisor of $p$, then $p$ doesn't divide $\binom{p}{q}$. Indeed, otherwise $\frac{(q-1)!}{p}\binom{p}{q}=\frac{(p-1)\cdots(p-q+1)}{q}$ would be an integer (but $q$ can't divide the numerator, since it is prime and doesn't divide $p-k$ for $0<k<q$).