It is easy to come up with objects that do not satisfy the Peano axioms. For example, let $\Bbb{S} = \Bbb N \cup \{Z\}$, and $SZ = S0$. Then this clearly violates the axiom that says that $Sa=Sb\to a = b$, unless we agree that $Z=0$, in which case what we have is exactly the standard natural numbers.
Similarly, consider the following structure: $$\Bbb T = \Bbb N \cup \{a, Sa, SSa, SSSa, SSSSa\},$$ where $S(SSSSa) = a $. This fails to satisfy the Peano axioms, but the failure is a little harder to find. The induction schema includes an axiom $$((0\ne SSSSS0)\land(\forall n. n\ne SSSSSn\to Sn\ne SSSSS(Sn))\to \\ \forall n. n\ne SSSSSn. $$
The andecedent is provable, and so the Peano axioms prove $\forall n. n\ne SSSSSn$, which rules out the possibility of $SSSSSa=a$.
Next, consider the structure $$\Bbb U = \Bbb N \cup \{ \overline 0, S\overline{0}, SS\overline{0}, \ldots\}.$$
This is ruled out by an induction axiom for the predicate $n=0\lor \exists m. n=Sm$; the predicate fails to hold for the element $\overline 0$.
Finally, consider $$\Bbb V = \Bbb N \cup \{ \ldots, N_{-2}, N_{-1}, N_0, N_1, \ldots\}$$
with the rule that $S(N_k) = N_{k+1}$. Here I wasn't able to find a theorem of PA that ruled out $\Bbb V$ as a model.
My questions (amended as per comments):
- I know at least two arguments that nonstandard models of PA must exist. But how can we be sure that some specific object is one of them, and some other object is not?
- Given some structure like $\Bbb U$ or $\Bbb V$ that is not a model of PA, is there a technique for finding a proof that it is not a model? Is there a technique for finding a particular theorem of PA that fails to hold in the non-model?
- I suppose this is treated in some elementary model theory textbook. Where can I find it?
There are some necessary conditions on the order type but Tenenbaum's theorem (addition and multiplication are non-computable in countable nonstandard models of PA whose elements are identified with $N$) sets a strong limit on how "specific" the object can be. Basically you would have to construct the object as a nonstandard model of PA, not start from a concrete combinatorial structure that plausibly could be a model and show that it is one.
It violates some necessary condition (usually order type, e.g., the example in the question with a cycle of length 5). According to the comment from Andres Caicedo, having an infinite ordinal be first-order definable in $U$ would make it not a model.
Kaye's book on models of PA.