If a set is a group of objects, then what is an object?
My best try at this is the following:
An object is anything that we can discuss or think about, separately from everything else. It is not necessarily a physical object, it can also be an imaginary object such as a green elephant.
Once we have defined an object, then it makes sense to talk about groups of objects, and this is where set theory begins.
From Joseph Shoenfield, Mathematical Logic (1967), page 238 :
Thus, if our choice is to have urelements, we can start with a collection of objects whatever: physical or abstract ones.
But if we start with physical objects, we are not licensed to assume the existence of infinitely many of them, while some sort of "axiom of infinity" is necessary for the development of "current" mathematics.