Suppose $V$ and $W$ are Banach spaces, $U\subset V$ is open, and $F:U\to W$ is a differentiable function. Then the derivative of $F$ is the map $$ DF:U\to B(V;W) $$ where $B(V;W)$ is the Banach space of continuous linear maps $V\to W$.
We say that $F$ is of class $\mathcal{C}^1$ at a point $x_0\in U$ if the mapping $$ U\ni x\mapsto DF(x) \in B(V;W) $$ is continuous at $x_0$; we say that $F$ is of class $\mathcal{C}^1$ on $U$ if $F$ is of class $\mathcal{C}^1$ at each point in $U$.
If $X$ is an arbitrary subset of the Banach space $V$ and $f:X\to W$ is a map, then we say that $f$ is of class $\mathcal{C}^1$ on $X$ if there exists an open subset $U$ of $V$ where $X\subset U$ and a function $F:U\to W$ of class $\mathcal{C}^1$ on $U$ where $F|_X=f$. (Informally, we can extend $f$ to an open set on which it is of class $\mathcal{C}^1$.)
See this answer for a function $f$ which is continuously differentiable at only a single point. Namely, if $g(t)=t^2\sin(1/t)$ for $t\in\mathbb{R}$ then the function $$ f(t) = \sum_{n\geq 1} \frac{g(t-1/n)}{2^n} $$ is continuously differentiable at $t=0$. However, $f$ has discontinuities arbitrarily close to the origin so $f$ cannot be of class $\mathcal{C}^1$ on any open set containing $0$.
That is, $f$ is a function which is of class $\mathcal{C}^1$ at $0$, but $f$ is not $\mathcal{C}^1$ on $\{0\}$.
This does not seem right to me. Of course, it is not "typical" for a function we encounter to behave this way. However, this example still bothers me. What can we do? Can we slightly modify the above definitions so that this does not happen? Is the answer I referenced somehow incorrect? (I couldn't prove the results he stated...)
The way I know it, a function is $\mathcal C^1$ on a set $X\subseteq V$ if it is $\mathcal C^1$ on the interior of $X$ and $\mathrm Df$ can be continuously extended to $X$. With this definition, your example function would be $\mathcal C^1$ on $\{0\}$, since the interior of this set is empty, and any function is vacuously $\mathcal C^1$ on the empty set. But this definition is really only interesting on sets with nonempty interior. The behavior of this definition on sets which don't fulfill $X=\overline{X^\circ}$, like $\{0\}$, is just a funny artefact. Also, it results in functions which are $\mathcal C^1$ on $\{0\}$, but not $\mathcal C^1$ in $0$, so the opposite dilemma of the one you mention.
For these reasons, it's generally best to restrict yourself to open sets or closures of open sets and not to worry about $\mathcal C^1$-ness on singleton sets. It wouldn't yield any big insights anyway. Then a definition would read as such:
Let $U\subseteq V$ be open. Then $\mathcal C^1(U,W)$ is the set of all continuously differentiable functions $U\to W$, and $\mathcal C^1(\overline U,W)$ is the set of all continuous functions $f:\overline U\to W$ for which $f\vert_U\in\mathcal C^1(U,W)$ such that $\mathrm D(f\vert_U)$ can be continuously extended to $\overline U$.