Since all norms on $\mathbb R^n$ are equivalent, the following question makes sense:
Can we define the notion of "differentiability" of a map $\mathbb R^n \to \mathbb R$ without refering to a norm at all? Can we define the derivative itself?
(That is, without mentioning any kind of norm, or a distance induced by it).
In fact, I guess that one could ask that even for $n=1$.
Here is an attempt to correct the below definition. Recall that $f$ is differentiable at $x \in \Bbb R^n$ if there exists a linear map $A:\Bbb R^n \to \Bbb R$ such that for $y \in \Bbb R^n$, the error function $$ \varepsilon(y) = f(x + y) - f(x) - A(y) $$ satisfies $\lim_{y \to 0} \frac{\varepsilon(y)}{\|y\|} = 0$. It is this limit condition, however, that needs to be encoded in a different way.
It is implied in this wiki page (linked in the comment below) that we can replace this limit condition with the statement that $\varepsilon:\Bbb R^n \to \Bbb R$ is tangent to 0. That is, for every neighborhood $W \subset \Bbb R$ of $0$, there exists a neighborhood $U \subset \Bbb R^n$ of $0$, a function $o:\Bbb R \to \Bbb R$ with $\lim_{t \to 0} o(t)/t = 0$, and a $\delta>0$ such that whenever $|t| < \delta$, $\varepsilon(tU) \subset o(t) W$.
We can make things a bit more concrete since the domain is simply $\Bbb R$. For every $\epsilon > 0$, there exists a neighborhood $U \subset \Bbb R^n$ of $0$, a function $o:\Bbb R \to \Bbb R$ with $\lim_{t \to 0} o(t)/t = 0$, and a $\delta>0$ such that whenever $|t| < \delta$, $\frac{|\varepsilon(tU)|}{o(t)} < \epsilon$.
Note: as Asaf notes in his comment, this is not a correct definition of differentiability. In particular, there are functions that satisfy this condition but fail to be differentiable.
We could stick to the product topology over $\Bbb R^n$ rather than considering norms.
Recall that we say that a function $f: \Bbb R^n \to \Bbb R$ is differentiable at $x \in \Bbb R^n$ if there exists a linear map $A: \Bbb R^n \to \Bbb R$ such that for every vector $v \in \Bbb R^n$, $$ \lim_{h \to 0} \frac{f(x + hv) - f(x) - hAv}{h} = 0. $$ With the topological definition of a limit, we might frame this as follows:
We can view the different norm-definitions as arising from different choices of neighborhood bases.
For example, we can derive the $\|\cdot\|_\infty$ (max-norm) definition as follows. Because the sets $U = (-\epsilon,\epsilon) \times \cdots \times (-\epsilon,\epsilon)$ form a neighborhood basis of $0 \in \Bbb R^n$, we can simplify this definition as follows:
If we instead consider the neighborhood basis of open spheres, we can derive the Euclidean-norm definition as follows: