OK, maybe the title is exaggerated, but is it true that the rest of math is just "good enough", or a good approximation of absolute truth - like Newtonian physics compared to general relativity? How do we know that our "approximation" is the right one? Another analogy: Fundamental physics is also not well-fundamented (where is the Higgs boson?) but most of the rest of the physics is on top of it, and it does its job well (it's a good-enough approximation).
Summary: According to the responses, math is indeed an imperfect domain, but can be seen as perfect for all practical purposes. In this case I wonder if math is indeed pure and identical across all possible universes. Maybe another universe comes up with a different set of axioms, more or less consistent than what we have now.
The first incompleteness theorem says that, under appropriate conditions, and for appropriate definitions of "true" and "provable," there exist statements that are true but not provable. This is interesting, but not a big deal. It is more or less a consequence of the existence of nonstandard models in first-order logic, which is an interesting feature of first-order logic, but mathematics is more than first-order logic.
The second incompleteness theorem says that, again under appropriate conditions, a sufficiently strong formal system can't prove its own consistency. Okay, this is kind of a big deal, but in practice it doesn't matter as much as it sounds like it does. Mathematicians don't actually do everything in a fixed formal system. There is a whole subject called reverse mathematics dedicated to finding the weakest formal systems that are capable of proving various things, which vary widely.
There is a possibility that ZFC, the formal system that sounds like it's what mathematicians work in (but isn't really), could be inconsistent. So what? If that ever happened we would just find another set of axioms to use. ZFC is a ridiculously strong formal system and in practice we never use its full strength, so it wouldn't really matter if it were inconsistent.
Mathematics is not the study of what formal statements can be proven in ZFC (although there are mathematicians who study this kind of thing). Proof: mathematics is thousands of years old, and ZFC is not.