Need help understanding a proof in a lecture

58 Views Asked by At

I cannot understand the proof. Going from 1st line to 2nd line in the proof, where does $\alpha$ come from? Why $w_{k+1} - w_k$ can be replaced by g?

The complete slide is here https://github.com/joanbruna/MathsDL-spring18/blob/master/lectures/bottou-02.06.2018.pdf.

enter image description here enter image description here

1

There are 1 best solutions below

0
On

slide 5, point 6 says that $w_{k+1} = w_k - \alpha_k g(w_k, \xi_k)$.

So $w_{k+1} - w_k = -\alpha_k g(w_k, \xi_k)$

This is applied twice in going from line 1 to line2 in the proof. Inside the norm, the $\alpha_k$ is pulled out (and is still squared) etc.