What is the Fréchet derivative?

Question

I'm sorry if I sound too ignorant, i don't have a high level of knowledge in math

---

I've been lately trying to understand the Frèchet derivative. I'm just starting Calc II, but I have a tiny grasp in multivariable calculus. That is, I understand that one can treat some of the variables as constants and get a directional derivative.

According to wikipedia, if the limit of the equation below as $h$ tends to $0$ is equal to $0$ then the function is said to be Fréchet differentiable at $x$.

$$\frac{||f(x+h)-f(x)-T(x)||}{||h||}$$

And as much as this can work as a definition, I'm still wondering what the Fréchet derivative actually is.

My confusion might also come from the fact that there are some ideas in multi-variable calculus I don't get, so I will try to summarize my questions below.

---

Q1. If I understood correctly, $f$ could be a multi-variable function, so how come we use only one $x$? Is it because $x$ is itself a multi-variable vector?

Q2. What is the meaning of $T(h)$? As in, what is it, and what is it doing in this equation?

Q3. Why does this equation -when $h$ tends to $0$- somehow tells whether a function is (Fréchet) differentiable or not? How does this equation relates to what a Fréchet derivative is?

---

Any help/thoughts would be really appreciated.

Answer 1

Q1: Yeah, $x$ is a vector; in general, we speak of $f : \mathbb{R}^n \to \mathbb{R^m}$, in which case $x \in \mathbb{R}^n$ and $f(x) \in \mathbb{R}^m$.

Q2: This was one of the coolest and hardest-to-grasp things (for me personally) when I first encountered multivariable. I'll try to sum it up here. In short: the derivative is not a number. Not anymore; in multivariable, the derivative is a linear transformation (and in single-variable, too, since this is a generalization; but in single-variable, the idea that the derivative is a number works fine - in multivariable, it doesn't make much sense). You can think of it as the best linear approximation to $f$; $T$, the derivative of $f$ at a point, is a linear function now.

Q3: The condition that this limit is zero means that this linear function $T$ is in fact a "first order" approximation. That is, $$ f(x + h) = f(x) + T(h) + \text{error term}, $$ where the error term tends to zero faster than $|h|$ does. This is just like the Taylor expansion in single variables.

Hope this helps. As a relation to single-variable, you can think of the usual derivative of some $f: \mathbb{R} \to \mathbb{R}$ as a linear transformation as follows.

The usual derivative of $f$ at some point $a$ is $f'(a)$, a number. In calc, you learn that the tangent line $y = f'(a)(x - a) + f(a)$ is the best linear approximation to the function $f$ near $a$. We can transform this into this new concept as follows: define $T : \mathbb{R} \to \mathbb{R}$ by $$ T(h) = f'(a)\cdot h. $$ Then $T$ is a linear transformation that takes a "vector" (in this case a single number) and returns $f'(a)$ times that vector. Note also that $$ \lim_{h \to 0}\frac{f(a + h) - f(a) - T(h)}{h} = \lim_{h \to 0}\frac{f(a + h) - f(a) - f'(a)h}{h} $$ $$ =\lim_{h \to 0}\frac{f(a + h) - f(a)}{h} - f'(a) = 0 $$ this last line following from splitting up the fraction, distributing the limit, and recognizing (by the "normal" definition of the derivative) that this limit is by definition f'(a).

Answer 2

The previous answer is fine. $T(h)$ is the linear term in the Taylor expansion of $f$ about $x$ in the direction $h$.

Perhaps this discussion might add to the previous answer with regards to Q3. The ratio that tends to 0 measures the error of the Taylor expansion $f(x+h) = f(x) + T(h) + R(x,h)$. Any theory of differentiation will have $R(x,h)\to 0$ as $h\to 0$, faster than $h\to0$, ($R = o(h)$, if you will) and the different kinds of differentiation (Gâteaux, Hadamard, Fréchet) differ only in the way in which $R$ is required to tend to 0. In particular, they ask for $R(x,th)/t\to0$ as $t\in\mathbb R $ tends to 0, pointwise for each $h$, uniformly in $h$ in compact sets, and uniformly in $h$ in bounded sets, respectively. (In finite dimensional spaces these last two coincide, but not in general.) By analogy with the 1 dimensional case, one wants $R(x,h)=o(h)$, to be of smaller order than $h$. But one cannot divide by vectors, and the different kinds of derivatives are different work-arounds. The Fréchet one corresponds to making $R=o(\|h\|)$.

For day to day practical work, Fréchet differentiability is the useful concept.