Is there a prime number $p$ with $p \mid \sum_{j=1}^{p-1} j!^2$?
I checked the primes upto $600\ 000$ without finding a solution.
Heuristic : If we can assume that the probability that $p$ is a solution is $\frac{1}{p}$ , then there should be infinite many solutions and therefore the desired prime should exist. But I guess this is not the case and maybe the special expression can easily be determined to be or not to be divisible by $p$.
Motivation : Such a prime number would prove that $1+2!^2+3!^2+\cdots n!^2$ can be prime only for finite many positive integers $n$ and also give an upper bound for the possible $n$. If the answer to the question is no , there is a chance that there are infinite many such primes.
Yes. By running a script which computes this sum mod $p$ through the first $100\,000$ primes, I found one solution: $p = 1\,248\,829$ ($96\,379$th prime).
UPDATE. This is also the only solution for the first $1\,000\,000$ primes.
UPDATE 2. If the probability that $p$ is the solution is indeed $1/p$, it follows from the Mertens' third theorem, that the probability to find the solution between $10^{m-1}$th prime and $10^m$th prime is asymptotically $1/m$. This could explain our serendipitous finding, but also discourages to look any further.