What is the relation between $\varepsilon$-$\delta$ and $dy$-$dx$ notations?

Question

For what I know, $dy$-$dx$ aren't real numbers, exist as convenient notations to capture our intuitions about infinitesimal increments, while $\varepsilon$-$\delta$ are real distances, saying that $f$ can be as close as we want to $L$ if $x$ is sufficiently close to $c$. The latter has a formal statement:

$$ \lim_{x \to c} f(x) = L \iff (\forall \varepsilon > 0)(\exists \ \delta > 0) (\forall x \in D)(0 < |x - c | < \delta \ \Rightarrow \ |f(x) - L| < \varepsilon)$$

While the former denote for $\Delta f$ and $\Delta x$ respectively when we arrive at $\Delta f=A\Delta x+o(\Delta x)$, which comes from the definition of derivative $f'=\lim_{\Delta x\to0}\frac{\Delta f}{\Delta x}$, and a chain of denotations $\varepsilon(\Delta x)=\frac{\Delta f}{\Delta x}-f'$, $o(\Delta x)=\varepsilon(\Delta x)\Delta x$ and $A=f'(x)$, with a note that $\lim_{\Delta x\to0}\varepsilon(\Delta x)=0$. As I understand here $\varepsilon(\Delta x)$ is simply a random unimportant convenient notation and has nothing relates to the infinitesimal above.

Yet looking at the graph on Wikipedia: (ε, δ)-definition of limit, I can't help but thinking that they are just one thing:

So what is the difference, and more generally, the relation between these two notations? Can I use $\varepsilon$-$\delta$ in a integral? And if $df=A\Delta x,dx=\Delta x$, then they should be real, right?

Answer 1

To begin with, your graph is wrong (or at best, misleading). The standard $\epsilon$-$\delta$ definition does not say that $f(x-\delta) = f(x) - \epsilon$ and that $f(x+\delta) = f(x) + \epsilon,$ although your graph asserts both of those facts.

To make a more representative graph, increase $\epsilon$ so that the horizontal lines do not all intersect the curve at the same places as the vertical lines. Put the lower horizontal line below the lowest intersection and the upper horizontal line above the highest intersection. That will be more representative of how to think about $\epsilon$-$\delta$ definitions.

Now to take a concrete example, instead of your arbitrary function drawn as a red curve, let's redraw the graph using the function $f(x) = x+1.$ And let's specifically choose $c=3,$ $\delta = 1,$ and $\epsilon=2.$

Then if we take the lower difference, we have $\Delta x = 2 - 3 = -1$ and $\Delta f = f(2) - f(3) = 3-4 = -1.$ But $f'(3) = 1.$ so we have $$ \frac{\Delta f}{\Delta x} - f' = 1 - 1 = 0. $$ We get the same result for the upper difference.

In fact you will have $ \frac{\Delta f}{\Delta x} - f' = 0 $ whenever you have a function $f$ whose slope is constant over the interval $[x-|\Delta x|, x+|\Delta x|].$ But $0$ is never a suitable value for $\epsilon$ in an $\epsilon$-$\delta$ definition.

The notation $\epsilon(\Delta x)$ seems unfortunate to me, since it actually has very little to do with the $\epsilon$ in the figure.

---

Added notes:

There are "upper" and "lower" differences because when you're taking a limit at a point inside your domain, you have to be able to approach it "from above" and "from below." For any $x,$ the "difference" is $\Delta x = x-c.$ An upper difference $\Delta x$ is positive and occurs when you come at $c$ from above, that is, $x > c.$ A lower difference $\Delta x$ is negative and occurs when you come at $c$ from below, that is, $x < c.$

In the example I gave, $c=3,$ so $x=2$ makes $x<c.$ We then have $\Delta x = -1,$ so $\Delta x$ is a "lower" difference.

Remember, $\epsilon$ in the $\delta$-$\epsilon$ definition is related to $\Delta f$ (through $|f(x)-L|<\epsilon,$ and all we need to do to satisfy the definition is to make sure that $|f(x)-L|$ is less than $\epsilon$; we don't have to make them equal. I wanted simple numbers in the example I gave, so I chose a relatively large value of $\epsilon$ ($\epsilon=2$), which gave me plenty of room to choose $\delta.$ And with $c=3$ and $\delta=1,$ if $|x - c|<\delta$ (that is, if $2 < x < 4$), we have $3 < f(x) = x+1 < 5.$ The limit we're going for is $\lim_{x\to3} x+1 = 4,$ so $L=4$ and $|x - c|<\delta$ implies $|f(x)-L|<\epsilon=2$ as the definition requires.

On the other hand, $\delta=2$ and $\epsilon=1$ gives us $1<x<5,$ $2<f(x)<6,$ and therefore $|x - c|<\delta$ implies $|f(x)-L|<2.$ But in this case $|x - c|<\delta$ does not imply $|f(x)-L|<\epsilon$; consider what happens when $x = 4.5,$ for example.

It may also be worth pointing out that your presentation of the $\delta$-$\epsilon$ definition and your graph were relevant to a limit of the function $f,$ but to find $\frac{dy}{dx}$ you take a limit of $\frac{\Delta f}{\Delta x},$ which is not $f.$ So if you're trying to relate anything about $\lim_{\Delta x\to0} \frac{\Delta f}{\Delta x}$ to the definition you wrote or to your graph, the symbols are just not going to match up. To apply the $\delta$-$\epsilon$ definition to $\lim_{\Delta x\to0}\frac{\Delta f}{\Delta x}$ rather than $\lim_{x\to c}f,$ you can write it like this:

$$ \lim_{\Delta x\to0} \tfrac{\Delta f}{\Delta x} = L \iff (\forall \varepsilon> 0)(\exists \delta > 0) (\forall \Delta x \in D') \left(0 < |\Delta x - 0| < \delta \implies \left|\tfrac{\Delta f}{\Delta x} - L\right| < \varepsilon\right)$$

where $\Delta x = x-c$ for some constant $c,$ $D'$ is the set of possible values of $\Delta x$ where $x$ is allowed to be anywhere in the domain of $f,$ and $\Delta f = f(c+\Delta x) - f(c).$

And the notation $\varepsilon(\Delta x),$ suggesting that the variable $\varepsilon$ in the definition above is somehow a function of the quantity $\Delta x$ in the definition? That's just wrong, wrong, wrong. Look at the definition again; it doesn't work that way at all.

Answer 2

The $\frac{dy}{dx}$ notation was first introduced by Leibniz 350 years ago. To Leibniz, $dx$ and $dy$ were infinitesimals. Some centuries later, mathematicians following Cantor, Dedekind, and Weierstrass (CDW) attempted to eliminate infinitesimals from mathematical practice starting around 1870. This was a time of formalisation and rigorisation and since they CDW were unable to formalize infinitesimals in a satisfactory way they felt there was no choice but eliminating them altogether. Less than a century later, Robinson was able to formalize infinitesimals to an acceptable level of rigor so in a sense Leibniz's ideas were retroactively vindicated over three centuries later.

In most (infinitesimal!) calculus courses infinitesimals are still not being used, in my opinion to great detriment of the students. So unless you are studying at places like Bar Ilan university where calculus with infinitesimals is taught, your professor will consider it an error if you view $\frac{dy}{dx}$ as an actual fraction.

The epsilon-delta definitions you mentioned are long-winded paraphrases of the original infinisimal definitions. For example, Cauchy defined continuity of $y=f(x)$ as follows: each infinitesimal increment $\alpha$ necessarily produces an infinitesimal change $f(x+\alpha)-f(x)$ of the values of the function. Due to unavailability of infinitesimals in a traditional CDW framework, the kind of definition you are likely to hear in your course is as you noted: for every epsilon there exists a delta such that if $x-c$ is less than delta then $f(x)-f(c)$ is less than epsilon.

Keisler's textbook does a fine job treating calculus with infinitesimals. Once you understand the key concepts like continuity and derivative via the more intuitive infinitesimal approach, you will be in a better position to follow the long-winded paraphrases, as well (but don't tell your professor about it :-).

Answer 3

The $\epsilon$-$\delta$ notation and the $dy$-$dx$ notations live in different spaces.

Specifically, the $\epsilon$-$\delta$ notation is native to the real numbers $\mathbb{R}$, and has to do with how real numbers are constructed and defined. Therefore, working with real numbers necessarily entails $\epsilon$-$\delta$ work. One axiom of the real numbers $\mathbb{R}$ is completeness. One way of formulating this axiom is that every Cauchy sequence converges. The definition of Cauchy sequence, and the definition of convergence, both involve $\epsilon$-$\delta$. Intuitively (and I think it is fair to credit this observation to Bolzano), as real numbers can never we written to full precision (they are constructed as limits), they are defined, constructed, and thought as being infinite sequences of increasingly precise approximations. This captures the intuition that we can write a real number to within as small an error as we want (e.g. we can write $\pi$ to as many decimal places as we want, subject to limitations of time and writing space), but never to full precision.

Conversely, in the $dy$-$dx$ notation, $dy$ and $dx$ exist only if we have infinitesimals. Infinitesimals do not exist in $\mathbb{R}$, so when we are working over the real numbers, $dy$-$dx$ is used as a shorthand for $\epsilon$-$\delta$ and has no intrinsic meaning. Indeed, over the real numbers the $dy$-$dx$ notation is misleading and treating $\frac{dy}{dx}$ as a fraction doesn't make sense and is an error. To write something rigourous, you strictly need to rephrase your expression in terms of $\epsilon$-$\delta$. However, if we are working over another ordered field which does have infinitesimals, such as the hyperreals, the $dy$ and $dx$ are infinitesimal numbers.