Reference request for Statistical Learning Theory

Question

I want to learn about Statistical Learning Theory. I have a strong background in measure theory, probability theory and functional analysis. What would be a good resource/book to start learning Statistical Learning Theory in the most general setting?

Thanks a lot in advance.

Answer 1

I think the following books can be very useful:

• Shai Shalev-Shwartz and Shai Ben-David. Understanding Machine Learning: From Theory to Algorithms.
• Mehryar Mohri, Afshin Rostamizadeh and Amit Talwar. Foundations of Machine Learning
• Michael Kearns and Umesh Vazirani. An Introduction to Computational Learning Theory. MIT Press, 1994.

Also, you can find the following courses helpful:

1. Understanding Machine Learning - Shai Ben-David
2. Computational Learning Theory by Varun Kanade

Answer 2

I would recommend "Elements of Statistical Learning" by Hastie, Trevor, Tibshirani, Robert, Friedman, Jerome. It covers a fair amount of material at a reasonable pace. Also, it is quite rigorous when necessary and on top of that, it provides beautiful images to clarify the concepts.

Answer 3

Just off the top of my head (I'll add other sources later on)

• Statistical Learning with Sparsity
• Martin Wainwright's statistical learning theory course
• Sham Kakade's statistical learning theory course
• Peter Bartlett's statistical learning theory course
• Percy Liang's statistical learning theory lecture notes

and references therein.