Is there a such thing called the "Classic Convergence Theorem"?

70 Views Asked by At

Has anyone heard of the "classical convergence theorem" before?

enter image description here

I tried searching online for this theorem but couldn't really find anything.

My Question: Does this theorem have a different name? Are there any theorems that state "stochastic gradient descent" will converge for either convex and non-convex functions?

The closest thing I found (that did not frighten me at first glance) was the following link https://gowerrobert.github.io/pdf/M2_statistique_optimisation/grad_conv.pdf, in which proofs are provided for stochastic gradient descent converging for convex functions, and page 6 makes some mention about non-convex functions.

Thanks!

Reference: