What actually is a differential?

Question

I am a bit confused about differentials, and this is probably partly due to what I find to be a rather confusing teaching approach. (I know there are a bunch of similar questions around, but none of them clarified my confusion).

When first meeting derivatives in calculus, the magic $d$ symbol first appeared in the Leibniz notation for derivatives as $\frac{df}{dx}$, and got told that it's a symbol for differentiation, not a fraction. The chain rule, saying that $\frac{df}{dx} = \frac{df}{du}\frac{du}{dx}$ was taught along the lines "looks like fraction simplification, but be careful".

Then integrals came around, with not much being said about the $dx$ at the end, until the substitution rule. Then, I made contact with differentials, but merely saying that when changing the variable, one needs also to change the differential $dx$ to $du=f'dx$. Now, it seems to me that $du=f'dx$ comes a bit from $f'=\frac{df}{dx}$.

When getting a bit into higher maths, there is more and more operations with functions/differentials. When doing surface areas, we talk about area differential, with $ds^2 = dx^2 + dy^2$, or differential of a multivariable function as $df = \sum\frac{\partial f}{\partial xi}dx_i$.

It seems like, in some cases, we do operate with the differential as its a simple real value, where the idea of an infinitesimal (basically $\Delta x = x - x_1$ as $x_1 \rightarrow x$.

Possibly the most important thing for me now, it seems like I can see the differential $df$ as a local, linear approximation of $f$ and $ \sum\frac{\partial f}{\partial xi}dx_i$ as a decomposition of the tangent along the basis directions.

But, at the same time, I remain stuck with the lack of an exact definition of the differential and a bit of fear of using it due to warnings such as "chain rule is not really a fraction simplification".

Answer 1

A very general definition of the differential is the Fréchet derivative:

Let $f:U\to W$ be a function where $(V,\|\cdot\|_V),(W,\|\cdot\|_W)$ are normed vector spaces and $U\subset V$ is open. $f$ is called differentiable at $x\in U$ if and only if there exists a bounded linear operator $A:V\to W$ such that $$\lim_{\substack{\|h\| \to 0\\h\in U}} \frac{ \| f(x + h) - f(x) - Ah \|_{W} }{ \|h\|_{V} } = 0.$$ Then $A$ is called the differential of f at $x$ and is often denoted by $\mathrm df(x)$.

Some important special cases (note that I use $x\cdot y$ exclusively for the product of two real numbers $x,y$ or for the product of a vector $x\in\mathbb R^n$ and a real number $y\in\mathbb R$):

1. For a differentiable $f:U\to\mathbb R$ (where $U\subset\mathbb R$ is open), $\mathrm df(x)$ corresponds to $f'(x)$ in the following way: $$\mathrm df(x)(y) = f'(x)\cdot y\in\Bbb R \text{ for all } x,y\in U.$$
2. More generally, for differentiable $f=(f_1,f_2,\dots,f_n):U\to\mathbb R^n$ (where $U\subset \mathbb R$ open), we have $$\mathrm df(x)(y)=f'(x)\cdot y\in\mathbb R^n\text{ for all } x,y\in U,$$ where $f'(x)=(f_1'(x),f_2'(x),\dots,f_n'(x))$.
3. If now $f$ depends on multiple variables, i.e. $f:U\to\mathbb R$ where $U\subset\mathbb R^n$ is open, then we have derivatives in multiple directions. For example, the derivatives in each cartesian coordinate are often denoted by $\frac{\partial f}{\partial x_i}$. Now, the differential of $f$ is the total differential. That is, $$\mathrm{d} f(x)(y) = \langle \operatorname{grad} f(x) , y \rangle,$$ where $\operatorname{grad} f(x)=\left(\frac{\partial f}{\partial x_1}(x),\cdots,\frac{\partial f}{\partial x_n}(x)\right)$.
4. Even more generally, if $f:U\to\mathbb R^m$, where $U\subset \Bbb R^n$ is open, then if $f$ is differentiable, $$\mathrm df(x)(y)=\operatorname{Jac} f(x)\cdot y\in\mathbb R^m,$$ where $$\operatorname{Jac} f(x):=\begin{pmatrix} \frac{\partial f_1}{\partial x_1}(x) & \frac{\partial f_1}{\partial x_2}(x) & \ldots & \frac{\partial f_1}{\partial x_n}(x) \\ \vdots & \vdots & \ddots & \vdots \\ \frac{\partial f_m}{\partial x_1}(x) & \frac{\partial f_m}{\partial x_2}(x) & \ldots & \frac{\partial f_m}{\partial x_n} (x) \end{pmatrix}.$$

About your notations: $\frac{\mathrm df}{\mathrm dx}$ is simply a different name for $f'$ where $f$ is as in 2.

$\frac{\mathrm df}{\mathrm du}=\frac{\mathrm du}{\mathrm dx}$ is a (maybe a little bit confusing) way of writing the chain rule:

If $f,g$ are as in 1., then $(f\circ g)'(x)=f'(g(x))\cdot g'(x)$ which can be reformulated as $$\mathrm d(f\circ g)(x) = \mathrm df(g(x))\circ\mathrm dg(x).$$

A very general case of the chain rule:
If $f:\mathbb R^m\to\mathbb R^k$ and $g:\mathbb R^n\to \mathbb R^m$ are as in 4., then $$\mathrm d(f\circ g)(x) = \mathrm df(g(x))\circ\mathrm dg(x),$$ i.e. $$\operatorname{Jac}(f\circ g)(x)=\operatorname{Jac}f(g(x))\cdot\operatorname{Jac}g(x).$$

Answer 2

Historically, Leibniz used differentials to capture the notion that we now convey through limits. Instead of talking about the limit of $\frac{\Delta f(x)}{\Delta x}$ as $\Delta x\to0$, he imagined $\frac{dy}{dx}$, where $dx$ was infinitesimally smaller than any real number and $dy$ was the corresponding change in $y$. Similarly, the integral went from $\sum f(x)\Delta x$ (an inaccurate sum of the areas of finitely many rectangles) to $\int f(x)\ dx$ (an exact sum of the areas of infinitely many "impossibly thin" rectangles).

The reason that we tell students, for instance, that the Chain Rule is not really a fraction simplification (even though it "just so happens" to work out that way) is that we formalized real analysis in the nineteenth century without infinitesimals but they didn't want to give up the familiar differential-based notation. So, once we formally prove with the limit definitions that things like the Chain Rule are valid, we can use our intuition-based notions of differentials to recall it. Or we can explore non-standard analysis where differentials are defined mathematical objects.

Answer 3

Originally, Calculus was based on infinitesimals. The idea is that $dx$ and $dy$ are infinitely small values. Unfortunately, Leibniz's concept of infinitesimal was not rigorously defined. Therefore, around the 1800s, Calculus was reformed from rigorous definitions, and the notion of infinitesimal got left behind. dy/dx remained because it was helpful, but was no longer considered to be an actual fraction. We went from infinitesimals being "ghosts of departed quantities" (Berkeley) to dy/dx being a ghost of a departed fraction.

However, in the 1960s, non-standard analysis showed that infinitesimals can in fact be rigorously defined and utilized as an extension of the reals (now called the "hyperreal" numbers). The essential tools was to add in a "standard part" function which essentially "rounded" a value that included infinitesimals to the nearest real number. This allowed infinitesimals to play the simultaneous zero/not-zero role that they always had in Calculus, but now defined in a rigorous way. I will use $\epsilon$ as the "base unit" of an infinitesimal.

To define a differential a little more rigorously, let's say that every equation/relation has a foundational independent variable that all the others are ultimately dependent upon, even if we don't name it. Let's call it $q$ for simplicity sake. So, if we have $z = y + x$ or something, we can assume that we are really saying $z(q) = y(q) + x(q)$. In such a scenario, when we write $dz$, what we really mean is $d(z)$, i.e., the differential function applied to $z$. That can be defined as $d(z(q)) = z(q + \epsilon) - z(q)$.

When defined in such a fashion, then you can see that differentials can easily be put into ratio with each other to make derivatives. I think this puts much of calculus on a much more intuitive setting, especially for those of us who are much more naturally inclined to basic algebraic thinking. A piece on how this can be put to good use is Simplifying and Refactoring Introductory Calculus.

By the way, the reason to be careful comes in the second derivative. If you want to treat derivatives as fractions, then you need to alter the notation for the second (and higher) derivative. If you think of the first derivative as a fraction, then, to get a higher derivative, the proper method to apply to get the derivative of the derivative is the quotient rule. This will yield the following notation for the second derivative: $$ \frac{d^2y}{dx^2} - \frac{dy}{dx}\frac{d^2x}{dx^2} $$ For more information on this, see Extending the Algebraic Manipulability of Differentials.