Currently I'm undergrad in Mathematics and confused about getting a Phd. I'm not interested in academia. I'm more interested in startups or jobs in the industry.
The only thing that is making me to do PhD is solving research problems. Say I'm reading a machine learning book or QFT book or a random thought in my mind and I find something(a problem) that can solved in a different way(maybe a new discovery!!) and more efficiently and it involves some research. Will I be able to do that research and solve that problem with only BS or do I need PhD?
Of course you can solve research problems in mathematics without a Ph.D.
That said, there are many reasons a Ph.D. is helpful (full disclosure: I just finished mine). Here are a few:
More knowledge. You learn a lot in grad school - new theorems, new proof techniques, new heuristics, etc. In the same way that it's easier to do research in math after college than after high school, it's easier after grad school than after college.
Advice. Doing research effecitvely is really really hard - there's a huge difference between doing well in even the most advanced math classes, and solving open problems. Having the focused help of someone who's done it before (your advisor) is extremely useful, as is being around other people in the same boat (listen to your fellow students!). This doesn't really exist, or at least not as easily, outside grad school.
Context. There's a lot of open problems out there. Some of them we have good reason to think will require fundamentally new techniques and huge amounts of work, and so probably aren't good targets; on the other hand, some problems that look extremely hard are very similar to ones which have been solved already, and so might be more accessible than they appear. One of the key roles of the advisor is to find good problems (which is not to say that students can't come up with their own problems - I certainly did occasionally - but this is a service your advisor can provide, and usually one's first problem is suggested by the advisor). Over time you learn how to discern good problems, but it's definitely a skill which is not innate.
EDITED TO ADD: