The efficient market theory which says that there’s nothing in the data, let’s say price data, which will indicate anything about the future, because the price is sort of always right, the price is always right in some sense. But that’s just not true. According to the famous mathematician, James Simons, there's some mathematical ways to predict prices. There's some anomalies in the data we could study. These are not big anomalies, since otherwise people could see them quickly and predict them. Then these anomalies have to be small and put together can allow to predict well the stock market. I know the mathematics behind is basically statistical, but I do not know where to start.
What types of anomalies can James Simons refer to build his models? How could we study, predict them mathematically?
Well, there isn't a single answer, it depends. For example Modern portfolio theory is a way to mathematically show that spreading the investment is safer (that work won a Nobel Prize). This is what most of the funds are doing.
Another way is to study correlations between different assets, events and assets etc. For instance if the price of petrol goes up, then the price of transporting the commodities will also go up, which will affect the prices of the transported commodities too. Sentiment analysis and correlation with stock markets was once a subject of serious studies, probably still is.
Hedging is another good strategy. For example a $-1$ or close to $-1$ correlation of the share prices of two rival companies (Apple and Microsoft was used once as an example). If one goes up, the other one goes down and vice-versa. Investing in both will reduce the risk of huge losses compared to investing only in one.
Looking for arbitrage possibilities, from differences in prices of the same assets on different markets (and markets these days tend to be well synchronised, but due to different software, networking and hardware related issues there are gaps sometimes, which are still successfully exploited even at the microseconds level) to simple rounding issues.
And I only listed 4. One important thing I should mention, finding specific examples of anomalies (rather than classes of anomalies) which work is quite hard. Those in books are almost surely outdated (this isn't suggesting not to read books, books are still good sources of inspiration). The new ones are being exploited and once made public - they aren't successful anymore. It's like a law of conservation of wealth - if more wealth appears somewhere, that means it disappeared from somewhere else. According to this rule, it's impossible for everyone to exploit successfully the same anomaly.