Through recursion, inversion, extension and all kinds of simple shenanigans the successor function leads to definitions of $+, \cdot, /, x^k, \exp, \sin, \partial, \int$ and other special functions. It seems to have uncanny utility. In particular, this set of functions happens to permit concise description of physics.
I ask, is this merely the result of directions math took in our history, or is there really something profound about $+$? Is $\mathbb N$ hard coded somewhere in the ZF axioms, or in the logic used to describe them? Among the uncountably many other functions, could there be another one, which is not trivially derived form $+$ but could match its utility?
$\mathbb{N}$ is "hard-coded" into logic: logical expressions are strings of symbols, and strings have lengths.
The theories of Peano arithmetic and string processing are essentially the same: with string processing, you can do arithmetic with lookup-tables and pushing symbols around. With Peano arithmetic, you can encode strings as digits of numbers.
An example of how this enters the picture can be seen with recursion. If you have an element $a$ of some set, and a function $f$ on that set, then you can talk about repeatedly applying $f$ to $a$. All of the expressions you can get look like $a, fa, ffa, fffa, \ldots$.
The individual terms can be labelled with natural numbers, counting how many $f$'s appear in the arithmetic expression.