I was wondering if there is any fast way to compute Fréchet/Gateaux Derivatives, or at least a reasonable guess in most cases, say from the usual derivatives table.
The Fréchet derivatives $\dfrac{\delta F}{\delta x}$of a functional $F: (X,||\cdot||)\mapsto \mathbb{R}$ is defined, if exists, as the functional $\dfrac{\delta F}{\delta x}: (X,||\cdot||)\mapsto \mathbb{R}$ such that $$\lim_{||h||\to 0}\dfrac{|F(x+h)-F(x)-\frac{\delta F}{\delta x}(h)|}{||h||}=0.$$
The Gateaux derivatives $\mathrm{d}F(x;h)$ of a functional $F: (X,||\cdot||)\mapsto \mathbb{R}$ in the direction $h\in X$ is defined, if exists, $$dF(x;h) = \lim_{\varepsilon \to 0} \dfrac{F(x+\varepsilon h)-F(x)}{\varepsilon}.$$
There is a way to quickly calculate Gateaux derivatives. It can also be used to calculate Frechet derivatives, if you are sure that a functional is Frechet differentiable, because in this case the Gateaux derivative also exists and coincides with the Frechet derivative.
Note that $$F(f+th): \mathbb{R}\to\mathbb{R},\ \text{fixed}\ f\in X,\ h\ \text{admissible}, t\in\mathbb{R}$$ $$\frac{dF(f+th)}{dt}\bigg|_{t=0}=\lim_{t\to0}\frac{F(f+th)-F(f)}{t}=dF(f;h)$$ Example: take the functional $$\int_0^1||f'(x)||^2dx,\ \text{restrict to}\ f(0)=a,\ f(1)=b$$ This is a convex functional over a convex set. For differentiable $h(x)\ \text{s.t.}\ h(0)=0,h(1)=0$: $$F(f+th)=\int_0^1||f'(x)+th'(x)||^2dx=\int_0^1||f'(x)||^2+2tf'(x)\cdot h'(x)+t^2||h'(x)||^2dx$$ $$dF(f;h)=\frac{dF(f+th)}{dt}\bigg|_{t=0}=\int_0^12f'(x)\cdot h'(x)dx=\int_0^12f''(x)h(x)dx$$ Note that $dF(f;h)=0$ when $f$ is the straight line from $a$ to $b$, this is the minimum of the functional.