I want to axiomatize "the concept of set" in my head, but every time I face some circular definition or intuition.
In predicate logic, we quantify over some "Universe of Discourse". Intuitively Universe of Discourse is a set, a collection. But then, in naive set theory or axiomatic set theory, we define "set" by using quantifiers (over Domain of Discourse)... Isn't it circular?
I mean, we assume that "Universe of Discourse" is some kind of "set", then by using first order predicate logic we define set again?
You're right to think that there is a seeming tension between the model theory of first-order logic and the interpretation of set theory. For instance, we know that first-order logic is sound for set-sized models - if $\phi$ is provable from $\psi$ and $\psi$ is true in a set-sized model $M$, then $\phi$ is also true in $M$. But that doesn't tell us whether first-order logic is sound with respect to non-set-sized interpretations, say the intended interpretation of set theory which has as its domain the collection of absolutely all sets.
This raises two questions:
(i) What are these non-set-sized interpretations, if not sets?
(ii) How do we know that the account of logical consequence in terms of set-sized models doesn't lead us astray? That is, how do we know that any formula true in all set-sized models is true in all interpretations and vice versa?
There are a few viable ways of answering (i) and an argument due to Georg Kreisel that answers (ii). A good exposition of these matters, plus an introduction to other issues in the area, can be found here.