Does set theory help understand machine learning or make new machine learning algorithms?

Question

When I was in a university, I didn't major in math but took some math classes. However, I dropped out of math classes pretty quick.

Some person recommended that I learn some set theory because it'll help me with Machine Learning . He recommended a book named Naive Set Theory by Halmos

Other people say it's not going to help much because set theory lives on a far higher abstraction level than mathematics used in machine learning do.

Since I don't know math well, I can't judge who's right.

Can anyone tell me how set theory is going to help me with machine learning ?

Answer 1

TL;DR : Set theory is a good thing to know , but there is more math out there which is actually critical for Machine Learning

You should probably learn Math in this order :

1) Pick up this OR this and flirt with the sections on proofs , set theory and its notations , functions , combinatorics , graphs ; till the time you feel comfortable solving a simple non-trivial problem . Do not pick up books like Halmos , please , -- not discrediting the book in any way - just not required here .

2) Learn Single Variable Calculus from here and here . Once you understand the intuition behind it , pick up a standard textbook and solve a lot of problems .

3) Learn a bit of Multi-Variable Calculus from here

4) Learn Linear Algebra from Strang . Ah ! the joys of learning from Strang . Seriously . You can do yourself no further good . Hop around to other MOOCs like this one .

5) Learn Probability and Statistics , IMO the most important math field required for ML . There are already pretty good resources and answers on the net , so will not repeat.

6) Learn how to apply statistics , probability theory to real-world problems intuitively by learning Data Mining and doing competitions on Kaggle

7) Take up Ng's Wonderful Course and embark on a hard yet fun journey of ML

Hope this helps :)

^{Disclaimer : I am not a ML Researcher , only a CS undergrad who has studied ML and is now working on a ML-based paper under a professor ; so it is possible that the order might vary according to your mileage}

Answer 2

Machine learning, if you really want to understand it rather than just believe the algorithms do what they are supposed to do, requires quite a heavy dose of mathematics. That mathematics is written in a language and that language is set theory a la Halmos (i.e., the naive kind). If you wish to properly understand the mathematics involved in ML, then it is a good idea to have a good grasp of the underlying universal language which is set theory. Halmos' book is quite short and does not go to the very high level of abstraction that set theory proper goes to. If ML is your goal, I'm would not say Halmos is a necessity, but it is certainly a worthwhile reading. Just don't expect it to translate into any algorithm.