On page of 22 Terry Tao's Analysis I, he writes (after presenting the Peano axioms):
Remark 2.1.14. Note that our definition of the natural numbers is axiomatic rather than constructive. We have not told you what the natural numbers are (so we do not address such questions as what the numbers are made of, are they physical objects, what do they measure, etc.) - we have only listed some things you can do with them (in fact, the only operation we have defined on them right now is the increment one) and some of the properties that they have. This is how mathematics works - it treats its objects abstractly, caring only about what properties the objects have, not what the objects are or what they mean. [...]
Remark 2.1.15. Historically, the realization that numbers could be treated axiomatically is very recent, not much more than a hundred years old. Before then, numbers were generally understood to be inextricably connected to some external concept [...] The great discovery of the late nineteenth century was that numbers can be understood abstractly via axioms, without necessarily needing a concrete model; of course a mathematician can use any of these models when it is convenient, to aid his or her intuition and understanding, but they can also be just as easily discarded when they begin to get in the way.
He then goes on to define sets axiomatically, and then, with the additional axiom that there is a set satisfying the axioms of the natural numbers, he constructs the rational numbers and real numbers.
However, I have often seen the rational numbers and real numbers also defined axiomatically--many times informally, where authors lay out certain properties of these numbers they will take for granted in their development of further results. Add to this list of objects often treated axiomatically (or informally axiomatically): points and lines, knots, symbols (when defining formal languages or formal polynomials, with unstated axioms being that they can be concatenated or exponentiated, etc.), functions, ordered tuples, and matrices, among others.
On the other hand, there are some objects which are typically constructed from the more primitive objects cited above, rather than being defined axiomatically. Among these are the complex numbers (defined as 2-tuples of reals, although axiomatic definitions simply stating, e.g., $i^2 = -1$ are also common), the extension field $\mathbb Q(\sqrt 2)$, or the $p$-adic numbers $\mathbb Q_7$, to name a few.
So, returning to my initial question,
- How is it determined which objects one should define axiomatically, taking them as primitive objects, and which ones constructively?
In Bill Thurston's essay, On Proof and Progress in Mathematics, he shares in the section entitled "What is a proof?":
Within any field, there are certain theorems and certain techniques that are generally known and generally accepted. When you write a paper, you refer to these without proof. You look at other papers in the field, and you see what facts they quote without proof, and what they cite in their bibliography.
Following along with this narrative, one answer to my question would be that the objects we take as primitive are just those that mathematicians in a given field generally take as primitive. But, even if we accept this as the case, I'm hoping to gain insight into the motivations behind the choices of which objects are primitive--I don't think they're entirely arbitrary.
Short soft answer to a soft question. Others may contribute more.
Where to start is an author's decision. You could teach analysis just with axioms for the real numbers. Tao chooses to start further back - essentially, as far back as we now know we can. He's decided that's a pedagogically sound way to teach real analysis.
My real analysis course in 1957 assumed the real numbers at the start of the first semester and constructed them from set theory and Peano at the start of the second. I remember struggling with integer arithmetic defined inductively when all I'd ever used induction for was proving things.