I am very confused about Lipchitz gradient/derivative. It's often used in machine learning proofs about gradient descent.
Are Lipschitz and epsilon delta proof essentially the same thing?
Lipchitz: A function $f:D\rightarrow \mathbb{R}$ is said to be a Lipschitz function, provided if there is a $L>0$ such that,
$$|f(x) - f(y)| \le L|x-y| \text{ for all } x,y \in D$$ Epsilon delta proof: For all $\epsilon>0$, there exists a $\delta>0$ such that
$$f(x)-f(y) < \epsilon \text{ if } |x-y| < \delta$$
The two ideas are saying the same thing: if $f(x)$ and $f(y)$ differ by this much, then $x$ and $y$ differ by that much. Am I right?
A related question posted here.
First, a counterexample: the function $f : [0,1] \to \mathbb R$ given by $f(x) = \sqrt{x}$ is continuous, and in fact uniformly continuous, but it is not Lipschitz.
Lipschitz continuity means that the difference ratio $$\frac{f(x)-f(y)}{x-y} $$ is bounded. So, it says something very specific about the comparison between the difference of $f(x)$ and $f(y)$ versus the difference of $x$ and $y$.
One way you can see that the square root function defined above is not Lipschitz is that the limit of difference ratios $$\lim_{x \to 0} \frac{f(x)-f(0)}{x-0} = \lim_{x \to 0} \frac{\sqrt{x}}{x} = \lim_{x \to 0} \frac{1}{\sqrt{x}} $$ is equal to infinity, and so the difference ratio $\frac{f(x)-f(y)}{x-y}$ is certainly not bounded.
The $\epsilon,\delta$ definition says nothing at all about the ratio $\frac{f(x)-f(y)}{x-y}$. Granted, though, it does say something about the difference of $f(x)$ and $f(y)$ versus the difference of $x$ and $y$, but what it says is quite a bit more vague than boundedness of the difference ratio.