This is a somewhat soft question, apologies if it turns out to be trivial/nonsensical.
Background: I was half-asleep one morning, not quite through my first cup of coffee, and thought about the "homomorphism" $\phi:\mathbb{Q}\to\mathbb{Z}/p$ given by $\frac{a}{b}\mapsto a\cdot b^{-1}$, which satisfies all the usual requirements ($\phi(ab)=\phi(a)\phi(b)$, $\phi(a+b)=\phi(a)+\phi(b)$ etc.) except that it isn't well-defined ($\phi(\frac{p\cdot a}{p\cdot b})$ is an obvious problem).
Now, supposing $p$ is very (e.g. uncomputably) large, we likely wouldn't ever in run into the concrete counterexample to $\phi$ being well-defined. Let's say we tried to build mathematics up from the bottom in the usual way (integers being equivalence classes of pairs of natural numbers etc.), only a clever demon gives us $\mathbb{Z}/p$ to start with instead of $\mathbb{Z}$. So, if we try to do the usual operations, e.g. $-3$ or $\frac12$, the demon gives us $-3 \mod p$ and $2^{-1} \mod p$ and tells us "this is $-3$ and $\frac12$".
Given that $p$ is far beyond our computational range, how can we detect if we've been duped?
"Let's say we tried to build mathematics up from the bottom in the usual way..."
Under all we have the natural numbers defined by the Peano axioms. No finite model verifies all the axioms.