I am looking at courses at different schools and I see optimization pop up here or there. I only understand optimization via maximizing and minimizing a function and particularly through machine learning where you are minimizing the loss function. What else is there to learn about optimization?
Additionally, if eventually, I want to implement and possibly develop my own ML algos, would optimization be something I should learn? I don't see myself ever doing proofs or solving derivatives by hand but numerical optimization techniques...I can see that being useful.
Anyways, what do you learn in those classes and in the broad picture, how important is optimization if your focus is on application rather than theory.