Convex optimization books, except with a focus on problem solving and formalism later?

Question

Let me start off by saying that I'm well aware of these books:

• Convex Optimization, Boyd and Vandenberghe
• Lectures on Convex Optimization, Nesterov
• The books by Bertsekas
• Convex Analysis by Rockafellar
• Convex Analysis and Linear Optimization by Borwein and Lewis

I'd also like to mention that my math background is by no means deficient: I have a math bachelor's degree, a master's in statistics, and am doing a two-semester sequence in measure theory. The advice seems to consistently be that if you know linear algebra and multivariable calculus, you should be able to handle this stuff.

I personally can't stand these books. Here's why:

• There is too much emphasis on formalism. Boyd and Vandenberghe, especially, seem insistent on defining term after term after term at you from the very beginning. Unless people who read this book are walking dictionaries, I don't see how one can reasonably learn from this book.
• There is too little emphasis on problem solving. Why do I want to learn optimization? So that I understand how to minimize/maximize an objective function with constraints. I understand the need for formalism, but the fact that this is front and center in every one of these books makes them extremely difficult for self study. Furthermore, I do not want to just learn how to formulate a problem as a convex optimization problem; I would like to at least know how to solve such problems by hand (or enough theory so that I can program such solutions).

Now that all of that is out of the way, to my main question:

Does there exist a convex optimization book which

1. has a problems-first emphasis, and if it provides formal results, does provide them but primarily in the context of actually solving a convex optimization problem?

2. is suitable for self-study, without supplementary material necessary? (I am not going to watch videos on top of a textbook; the book should be able to stand on its own.)

3. is not just a definition-theorem-proof-type of book?

Please note that I am not looking for a linear optimization book, but a convex optimization one.

Answer 1

Given your background in statistics perhaps you could look at Online Convex Optimisation, which is an area that combines convex optimization and regret minimisation in sequential learning problems. The resources in this area usually cover a lot of convex optimisation theory that is useful for solving actual problems in that area. Online learning is an area with a lot of open problems and has been an active area for research, but also covers a lot of practical problems you'd face in an industry setting, e.g. portfolio optimisation, recommender systems.

Some links:

1. Online Convex Optimization Tutorial and list of resources
2. Elad Hazan's book on online convex optimization
3. Online Learning notes by Francesco Orabona

Answer 2

I would recommend to give Boyd and Vandenberghe another try, but this time start directly from the second part of the book (Applications, Chapter 6-), leaving the first part as reference. The covered applications are from machine learning and statistics, which is quite generic.