Can you suggest some good optimization books?

Question

I am looking for optimization books. Can you suggest some good materials?

First, I started with Convex Optimization by Stephen Boyd & Lieven Vandenberghe, but I don't like it because they don't give examples of proofs and techniques, only theory and talking. I need some classical books, for example I like books such as: Zorich, Kreyszig, Kolmogorov and Fomin. Please suggest books that have a similar style. Especially I need books to pursue research in Reinforcement Learning.

Answer 1

Try these books $(1)$ N. S. Kambo, Mathematical Programming Techniques, East West Press, 1997 and $(2)$ E.K.P. Chong and S.H. Zak, An Introduction to Optimization, 2nd Ed., Wiley, 2010.

Answer 2

Belegundu and Chandrupatla, Optimization Concepts and Applications in Engineering.

Answer 3

Since you've mentioned you wish to take up research in reinforcement learning, I'm assuming by "optimization" you mean both convex and non-convex optimization. I'd suggest you the following depending on your level of understanding:

1) Introduction to Linear Optimization by Bertsimas and Tsitsiklis: A good starting book on linear optimization that will prepare you for convex optimization.

2) Introductory Lectures on Convex Optimization by Yurii Nesterov: Nesterov is a living legend in the field of convex optimization. You might have heard his name from the famous Nesterov Momentum technique. Be warned, this book isn't light in its usage of mathematics. An even math-heavier version of this book titled Lectures on Convex Optimization is also published by Springer.

3) Optimization for Machine Learning by Sra, Nowozin and Wright: While I haven't taken a look at this book, I'm told it is high quality and covers both convex and non-convex optimization.

4) Non-Convex Optimization for Machine Learning by Jain and Kar: This monograph is a big gun, when it comes to non-convex optimization, and is freely available online. I'm not sure how much of the material would be helpful to you, but it should serve as a good reference. There are not many books which specifically cater to non-convex optimization, this book is one of the first dedicated texts I could find. But then again, this isn't exactly an easy read.

Happy learning.

Answer 4

Optimization by Vector Space Methods by Luenbeger, and Non-Linear Programming by Bertsekas.