I'm trudging through Barwise's Handbook of Mathematical Logic and came across this:
Here is a good exercise. A ring $\mathfrak{R}$ is a principal ideal ring if $\mathfrak{R}$ is a model of the second-order sentence: $$\forall I[I\text{ an ideal}\rightarrow\exists x \thinspace \forall y (I(y)\leftrightarrow\exists z(y=zx))]\text{.}$$This cannot be expressed in first order logic. Indeed, a simple compactness argument shows that there is a ring $\mathfrak{R}$ with the same first order properties as the ring $\mathbb{Z}$ of integers (written $\mathbb{Z}\equiv\mathfrak{R}$) but where $\mathfrak{R}$ is not a principal ideal ring.
Not a (satisfying) solution. Following the prompt, take $\mathbb{Z}$ and add to it an additional constant symbol $\mathbf{x}$. Along with the true sentences of $\mathbb{Z}$, throw in "$\neg\exists\thinspace n\in \mathbb{N}[n1=\mathbf{x}]$" by including the infinite family of sentences $n1\neq\mathbf{x}$. Now, take some finite subset of all the sentences. The polynomial ring $\mathbb{Z}[x]$ is a model of the subset. It is, a bit disappointingly, also a model of all the sentences in the set. The ideal generated by $2$ and $3x^2$ is not a principal ideal, so $\mathbb{Z}[x]$ is not a principal ideal ring.
(I think) this answer is correct as it goes, but it's certainly a bit of a let down. First, since the purpose to which compactness arguments are put to throughout the book is proving the existence of a model with a certain property, a proof which relies on already having such a model at hand seems more than a bit superfluous. Moreover it's a pretty large break from all the examples given in the book in that, for a particular finite subset, the model provided is a finite one. That may not be true in this case, since it seems like much less work to use normal polynomial multiplication than multiplication modulo some non-constant polynomial, but even so there's something of the kind missing here. What's a better way of thinking about this problem?
If I understand a part of your objection correctly, it asserts in particular that $\mathbb{Z}[x]$ is elementarily equivalent to $\mathbb{Z}$. That is not the case. For example, in $\mathbb{Z}$ it is true that for any $w$, one of $w$ or $-w$ is a sum of $4$ squares. This is not true in $\mathbb{Z}[x]$. Take for example $w=x$.
Now we show how to produce a structure elementarily equivalent to $\mathbb{Z}$ which is not a principal ideal domain.
Let $T$ be the set of all sentences true in $\mathbb Z$. To the language of ring theory add infinitely many constant symbols $a_1,a_2,a_3, \dots$. To $T$, for each $i$ add axioms that say that $a_i$ is positive, and for every natural number $n$, that $a_{i+1}\lt a_i^n$. So we write $a_{i+1}\lt a_i\cdot a_i$, $a_{i+1}\lt a_i\cdot a_i \cdot a_i$, and so on. The resulting theory $T'$ is consistent. In any model of $T'$, the ideal generated by the interpretations of the $a_i$ is non-principal.
Added: We can do it with a single constant symbol $C$, by adding axioms that say that $C$ is positive and divisible by $2$ and by $4$ and by $8$ and so on. Let $M$ be a model, and let $c$ be the interpretation of $C$ in $M$. Then the ideal generated by $c,c/2,c/4,\dots$ is non-principal.