In Dabrowski's paper, he showed that it would follow from the abc conjecture that the equation $$n!+k=m^2$$ has a finite number of solutions $n, m$ for any given $k$ which was my motivation to find solutions for different values of $k$.
Using PARI/GP, I observed that for any $k \gt 1$, if $n!+k$ is a square, then $n \le k$. I didn't find any counterexample in my search that covered a range of $k\le 2500$ and $n\le 10^4$ for each $k$.
Questions:
$(1)$ Can we prove that for any $k \gt 1$, if $n!+k$ is a square then $n \le k$, thereby restricting Dabrowski's original statement?
$(2)$ If false then what would be the smallest counterexample?
Update 1: This also seems true for $n!-k$, when $k\gt 2$.
Update 2: After some more testing on PARI, I conjecture that for any $k \gt 3$, if $n!+k$ is a perfect power, then $n\le k$. This also seems true for $n!-k$.
Claim: If $k$ is prime and $n!+k$ is a square, then $n \le k$.
Proof: Clearly $k$ can't be $2$ (mod $4$ considerations), so $k$ is odd and then, by mod $4$ considerations, is $1$ mod $4$. Then $n!+k = m^2$ implies that for each odd $p \le n$, $(\frac{k}{p}) = 1$, which implies $(\frac{p}{k}) = 1$ by quadratic reciprocity (since $k$ is $1$ mod $4$). Also, $n!+k = m^2$ directly implies $k$ is $1$ mod $8$, so $(\frac{2}{k}) = 1$. Therefore, if $n \ge k$, then each $p \le k$ has $(\frac{p}{k}) = 1$, and thus by multiplicativity, we get $(\frac{m}{k}) = 1$ for each $m \le k$, which is impossible, since there are $\frac{k+1}{2}$ quadratic residues mod $k$. $\square$
.
WE Tutorial School's answer below shows that $n \le k$ if $k$ is a nonsquare composite. The argument is as follows. Note $k \mid \frac{n!}{k}$, since if $k = rs$ for $1 < r < s < k$, then $r,s$ appear in $n!$ as well as $k$ (assuming $n > k$). So $\frac{n!}{k}+1$ is relatively prime to $k$, but then that $k(\frac{n!}{k}+1) = n!+k$ is a perfect square means that $k$ must be a perfect square, which we assumed it isn't.
.
This leaves open the case of $k$ a perfect square. It should be noted that answering question (1) in the affirmative would then improve Dabrowski's result, so seems hard.