Whilst I was trying to think of a proof for the chain rule for Fréchet derivatives, I realized it looks very similar to the naturality axiom for functors. (Except for the need to specify points for the derivative.)
$$ D(f \circ g)_{p} = (Df)_{g(p)}\circ (Dg)_p \sim F(f \circ g) = F(f) \circ F(g) $$
Is this just a coincidence? Or is this hinting at some deeper meaning of derivatives?
On
If you're not ready to think about manifolds, you can still think of the derivative as a functor on the category whose objects are $\mathbb{R}^m$ and whose morphisms are smooth maps. The (total) derivative of $f:\mathbb{R}^m\to \mathbb{R}^n$ is $Tf:\mathbb{R}^{2m}\to \mathbb{R}^{2n}$, sending $(\vec x,\vec y)$ to $(f(\vec x),Df_{\vec x}(\vec y))$. This is the specialization of the notion of "tangent bundle" mentioned in the comments to Euclidean spaces.
The way I prefer to say things is this. There is a functor from, say, the category of pointed smooth manifolds to the category of vector spaces, which takes a smooth map $f : (M, m) \to (N, n)$ (so $m \in M, n \in N$ and $f(m) = n$) to the derivative $df_m : T_m(M) \to T_n(N)$. The fact that this respects composition is precisely the chain rule. There are many variations on this.