Understanding the Frechet derivative

Question

What is the relationship between the normal 'high school' concept of a derivative and a Frechet derivative? According to wikipedia the Frechet "extends the idea of the derivative from real-valued functions of one real variable to functions on Banach spaces."

Suppose we have $f:(\mathbb{R},\|\cdot\|_{\mathbb{R}})\rightarrow (\mathbb{R},\|\cdot\|_{\mathbb{R}})$ where $\|x\|_{\mathbb{R}}=|x|$ and $f(x)=x$. I think $(\mathbb{R},\|\cdot\|_{\mathbb{R}})$ is Banach so I feel that ideally the Frechet derivative of $f(x)$ at 0 should be equal to our standard derivative at $0$ which is $f'(0)=1$. But $$\lim_{h \rightarrow 0} \frac{|f(x+h)-f(x)-1|}{|h|} \neq 0$$ So my question is how does the Frechet derivative extend the idea of a derivative from real-valued function of one variable?

Answer 1

The only difference is that you've moved everything to one side of the equation.

$$ f'(a) = \lim\limits_{h \to 0} \frac{f(a+h)-f(a)}{h}$$

becomes

$$ 0 = \lim\limits_{h \to 0} \frac{f(a+h)-f(a)}{h} - \frac{hf'(a)}{h} $$

so that

$$ 0 = \lim\limits_{h \to 0} \frac{f(a+h)-f(a)-f'(a)h}{h}$$

chucking in some absolute values doesn't change anything

$$ 0 = \lim\limits_{h \to 0} \frac{|f(a+h)-f(a)-f'(a)h|}{|h|}$$

The utility of defining derivatives this way is that it extends to situations other than that of functions of one variable. Let's recast this once more and instead of sending $h$ to $0$ we can equivalently send $x$ to $a$ ($h=x-a$). Then we get

$$ 0 = \lim\limits_{x \to a} \frac{|f(x)-f(a)-f'(a)(x-a)|}{|x-a|}$$

Now if you replace $x$ and $a$ with vectors, $f$ with a function from vectors to vectors and think of the absolute value as the length of a vector, we have a perfectly reasonable definition for a derivative. Well...except for this "$f'(a)(x-a)$" business.

We need to replace the "number" $f'(a)$ with a linear operator (or a matrix = Jacobian) and then everything makes sense.

By the way, this is my preferred way of presenting derivatives in multivariable calculus. We see the derivative as being a linearization which well approximates our function: $f(x) \approx f(a)+f'(a)(x-a)$ (the tangent). When $f$ is a scalar-valued function, $f'(a)$ is just the gradient. Also, we get that approaching this multivariate limit along coordinate axes reduces to partial derivatives. This then explains why partials (and in fact all directional derivatives) can exist at a point even when a function is not differentiable (a limit can exist along all lines but still fail to exist).

Answer 2

It should be $h$, not $1$.

When we have function from $\Bbb R$ to itself, we have $f(x+h)=f(x)+f'(x)h+o(h)$ whereas in the general case, $f(x+h)=f(x)+D_x(f)(h)+o(\|h\|)$. So $D_x(f)(h)=f'(x)h$ which equals $1\times h=h$ in your case.