Some physicists have argued that (other things being equal) a beautiful theory is more likely to be true than an ugly one. Dirac famously took this view: "A theory with mathematical beauty is more likely to be correct than an ugly one that fits some experimental data."
It's not uncommon for mathematicians to talk about beauty in mathematics. (I always think of Hardy, "there is no permanent place in the world for ugly mathematics.") But are there mathematicians who think (as some physicists do) that beauty (or elegance, or whatever) is evidence of truth?
I've done some Googling, and I've found many discussions of beauty in maths, but none of the things I've seen so far deal with my question directly.
It's empirically true that reality has so far cleaved rather well to neat models. See The Unreasonable Effectiveness of Mathematics in the Natural Sciences for much more. Since it's always been true so far that "every mystery we've ever solved has turned out to be
not magicnicely modellable", we often tend to treat this as evidence that any other given mystery will be nicely modellable.However, all our models so far have been very much approximations, and we know that they're not Truly Correct because they're not all compatible with each other (c.f. quantum mechanics and general relativity, both excellent but incompatible models). So it's quite possible that reality does not in fact admit any nice compact description.
Note also that there's a huge selection effect going on here: we know how to solve physical problems using mathematics, so of course the problems we have solved are overwhelmingly done using mathematics!
I will note, as a software engineer, that coming up with a solution to a software problem that in some sense has mathematical beauty (e.g. it has no edge cases) does seem to be a pretty good indicator that you've found The Right Answer. In my experience, such a solution will be extensible, low in bugs, require fewer rewrites, and so on, and it often turns out to model well entirely new requirements that I hadn't even thought of. But I would argue that this is more likely because an answer with a neat mathematical model is simpler and has fewer moving parts, all of which are more easily described and more easily proved correct, than because there is something inherently "correct" about mathematics as applied to the real world.