Has anyone heard of the "classical convergence theorem" before?
I tried searching online for this theorem but couldn't really find anything.
My Question: Does this theorem have a different name? Are there any theorems that state "stochastic gradient descent" will converge for either convex and non-convex functions?
The closest thing I found (that did not frighten me at first glance) was the following link https://gowerrobert.github.io/pdf/M2_statistique_optimisation/grad_conv.pdf, in which proofs are provided for stochastic gradient descent converging for convex functions, and page 6 makes some mention about non-convex functions.
Thanks!
Reference:
- https://youtu.be/a4xaK0dsZ_c (@6:40)
