Take $X=\operatorname{Spec}(k[x,y]/(y^2-x^p+t))$ with $k=\mathbb{F}_p(t)$ and $p\neq2$.
The Jacobian ideal is $J=(y,y^2-x^p+t)=(y,x^p-t)$ which is maximal ie a closed point of $X$ so Jacobian criterion work to say that $\mathfrak{p}=(y,x^p-t)$ is singular.
Let's $A=k[x,y]/(y^2-x^p+t)$ then in $A_\mathfrak{p}$ one have $\mathfrak{p}A_\mathfrak{p}=(y)$ because $x^p-t=y^2\in(y)$ so $\mathfrak{p}A_\mathfrak{p}$ has so many generator as $\dim A_\mathfrak{p}$ (which is 1 because $\mathfrak{p}$ is maximal in $A$ so $\dim A=\dim A_\mathfrak{p}$). So $A_\mathfrak{p}$ is regular ie $\mathfrak{p}$ is regular.
Where is my stupid mistake?
The point is that, as Mohan pointed out, the Jacobian criterion checks whether $X\to\mathrm{Spec}(k)$ is $\underline{\text{smooth}}$. The notion of regularity is related, but not identical.
In fact, we have the following:
In particular, if $k$ is perfect then $X$ is smooth over $k$ if and only if its regular. The necessity that $X_{k^\mathrm{perf}}$ is illustrated by your example and, in fact, is even the example they give on The Stacks Project to highlight this issue (see Tag038Y). In fact, you can check that for your $X$ you even that $X_{k^\mathrm{sep}}$ is regular!