From what I've understood, Naive Set Theory does not mix very well with classical logic because it eventually boils down every statement to contradiction.
This is not to say Naive Set Theory is strictly useless, as long as you're willing to work with more exotic logic.
Pretty much all of the mathematics I've learnt is based on, as far as I know, the ZFC axioms. I understand that there are other alternatives to ZFC, but perhaps, have we figured out a way to ensure that something like Russell's Paradox will not arise in an arbitrary axiomatic foundation for sets? (when "scrutinised" under classical logic)
This is very close to a duplicate of various questions, but I can't find an exact dupe at the moment.
Ultimately we are not certain of this. Indeed, although it's very rare there have been quite excellent mathematicians who suspected that $\mathsf{ZFC}$ is inconsistent after all (such as the logician Jack Silver). And this is all leaving aside the second incompleteness theorem. Personally, while I am as certain of the consistency of first-order Peano arithmetic as I am of the fact that I have two hands, I would merely be profoundly disturbed to learn that $\mathsf{ZFC}$ is inconsistent.
Fortunately, $\mathsf{ZFC}$ is so ridiculously overpowered that its inconsistency wouldn't really spill over to the rest of mathematics too much. For the vast majority of mathematical practice, galactically weaker theories like $\mathsf{ZC}$ are enough.
Incidentally, the question of exactly how much "axiomatic overhead" is needed for various parts of mathematics is studied rigorously in Reverse Mathematics. It turns out that a huge amount of mathematics can be developed in $\Pi^1_1$-$\mathsf{CA_0}$, which is a tiny fragment of $\mathsf{Z_2}$ which is itself a tiny fragment of $\mathsf{ZC}$. The primary difficulty with RM's approach is the restricted language, which makes lots of "higher-type" mathematics (e.g. measure theory, topology, ...) hard or impossible to treat faithfully, but there has been some recent work in improving this situation (see e.g. here).