The question is motivated by this
As $6! = 5\cdot 12^2$, in order to prove that $6!$ is not a square, we need to know that $5$ exists between $3$ and $6$. So Chebyshev's theorem seems necessary here. However, I believe this is the only case (other than 2) where $n! = p m^2$ with $p$ prime; the reason must be also in some result of Chebyshev kind: there are at least two different primes between $n$ and $\frac {n}{2}$ or something similar. But I have no proper proof.
There is another solution :
$$10!=7\times 720^2$$ This is almost surely the largest one.
This PARI/GP program
shows that the only solutions upto $n=2000$ are $2,6,10$. To prove the conjecture, it would be enough to show that for every even number $n\ge 2000$, there are at least two primes $p,q$ with $\frac{n}{2}<p<q<n$.