How can it be that mathematics, being after all a product of human thought independent of experience, is so admirably adapted to the objects of reality$?$
The above quotation is by Albert Einstein. I was wondering what did he really mean by that$?$
So if we consider a mathematical system or a mathematical model corresponding to a physical system and then we develop that mathematical system independent of that physical system, then still those two systems could be related$?$ Is it where the power of mathematics lies$?$
I don't think anyone will be able to say with certainty exactly what Einstein meant, but we can take a stab at it, and in my opinion I think that what you said is essentially what he was trying to get at. I think it's what makes Mathematics so amazing. It's like a magic trick, but the magic doesn't wear off "just because you know how it's done".
Mathematics for many, many years now has been independently developed from the Sciences. Originally, Mathematics would have been deep-rooted in classical, Physical thinking - for example, the invention of the Natural Numbers. It's very easy to see the usefulness in being able to count objects for trading things, for example (and hence so would being able to add and subtract!). But the idea of a number like $2$ "existing" as a separate thing would not have been a thing at all - it would have been purely a label. No one can go and point out the number $2$ and say "there it is!". It really is purely in your mind - a construct.
Yet we still develop more and more theories, purely on imaginary objects that exist only within our imagination - and yet a lot of it can be used to describe the world around us. I like to think of Mathematics as a sort of "artificial sense" - you can see, hear, taste, smell - in the same way Mathematics allows you to logically deduce things about your environment. Things that are not calculable from our standard Biological senses. It really is quite absurd when you think about it in this way:
$${\text{Reality}\rightarrow\text{Imaginary place in your mind}\rightarrow \text{Reality}}$$
A counterargument to this is "well, Mathematics is Fundamentally deep-rooted in reality, and so are we - so of course it works out nicely!". Which is true. But personally, this doesn't deduct from the beauty of it at all for me.
That all being said, Mathematics is separate from Physics. As Mathematicians, we consider problems and create things all the time that are not necessarily immediately Physically applicable. We do this because (a) it can lead to the creation of more Mathematics, and this could later be helpful in the real world (this is the beauty we believe Einstein is referring to) and (b) because it's a challenge, and physically applicable or not - the results are usually still beautiful.