In my textbook A Course in Mathematical Analysis by Professor Garling, he utilized Axiom of Regularity to prove one of the properties of models of natural numbers, and afterwards he never mentions Axiom of Regularity anymore.
I know that we need other axioms to resolve some paradoxes such as Russell's paradox. And this paradox is mentioned frequently in Analysis textbook. But I have not observed the same motivation for Axiom of Regularity.
I think there must be another deeper motivation for adopting this axiom into ZF, but i'm just recently exposed to set theory. Please give me some explanations!
The axiom of regularity is hard to motivate from a somewhat naive point of view.
It comes into play in full force when you are studying set theory in a more advanced level, since it is effectively equivalent to saying that the universe of sets can be internally generated from the empty set using power set and unions.
The thing, however, is that as far as "everyday mathematics" is concerned, you are just looking for a set of size continuum, or a set which is countable. Because once you have a countable set, you can endow it with the structure of the natural numbers, and once you have a set of size continuum, you can make it "look like" the real numbers, or the complex numbers, or any other set you wish with that cardinality.
At least for these two cardinals, you can always find sets whose existence is compatible with the axiom of regularity (namely, $\omega$ and $\mathcal P(\omega)$). And if you also assume the axiom of choice, then you can ensure that every set can be replaced by a set of ordinals, and so a set whose existence is compatible with regularity.
So really, there is no reason to insist otherwise. It's not only that, though. You want your sets to have some sort of a grounding mechanism. Because that's how our brains think.
We accept the real numbers because they are generated by the rational numbers. We accept the rational numbers because they are generated by the integers. We accept the integers because they are generated by the natural numbers. And the natural numbers accept because they make sense to us.1
Similarly with sets, the empty set makes sense to us. It's a set with no elements. So, if a set makes sense, and the power set operation makes sense, that means that $\mathcal P(\varnothing)$ also makes sense. And so on. The axiom of regularity, essentially, says that "all sets make sense", in a technical way. Which is the philosophical and practical motivation for it.
Let me just finish that some people might claim that it is motivated in solving the Russell paradox that not every collection is a set. For example, if $\{x\mid x=x\}$ is a set, then it is an element of itself, which is impossible by the axiom of regularity. So the Russell paradox has been avoided!
Alas, the Russell paradox is avoided without assuming regularity just as well. It is true that regularity provides a slightly shorter proof, but it serves as a red herring.