It seems to me that in trying to create a foundation of mathematics, mathematicians are trying to create a formal system that models the language used in what I view as common-sense, real mathematics. Whatever the number $1$ is, I doubt it's $\{\{\}\}$, no? The set containing the empty set is merely a representation of $1$ in this formalism. So then is a foundation of mathematics merely a formalism used to model 'real' mathematics? If not, what is it?
I thought I was in on a little-known secret when I was (perhaps not intentionally) lead to believe that $1$ is the same as $\{\{\}\}$, but now I see how preposterous a claim that is, especially considering that the empty set is probably but a linguistic construct.
If the answer to my first question is 'yes', then presumably the reason for the foundation is to check it for self-consistency, for if the foundation is self-consistent, the thought process probably goes, then 'real' mathematics is self-consistent, yes? How can one know that the foundation (always) parallels 'real' mathematics? How do we know that the two discourses are isomorphic? If we can't know then why bother with a foundation in the first place? If we can know then presumably it's due to some metalanguage, but then... is the metalanguage self-consistent? How do we know? Isn't this just kicking the can down the road?
There are various approaches to a foundation of mathematics. Russell and Whitehead in Principia Mathematica defined $1$, for example, as the set of all singletons, not just $\{\{\}\}$.
Gödel showed there are limits to what we can prove with formal systems, e.g. there are true statements of arithmetic that are unprovable.
Research on a foundation for mathematics using axiomatic systems, has led to some major results in logic and set theory. From a more practical perspective, first-order logic and the Simple Theory of Types are applied in program verification and hardware verification, e.g. Hoare Logic, HOL and Isabelle.