I've struggled for a couple of weeks to figure out the question I really want to ask, but without any success, so I'm just putting out this rather confused question out in hopes that someone can help me. The short version is:
Why does the harmonic series diverge so quickly?
Before you jump to answer this (or to tell me it does not diverge quickly) please read the rest of the post.
Let's say as usual that $$\zeta(r) = \sum_{i=1}^\infty i^{-r},$$ where I will only be considering real $r$. For $r=0$ the series obviously diverges and for $r=2$ it is well-known to converge. In fact it converges if and only if $r>1$.
It's clear that as $r$ increases from $-\infty$ to $1$ the series $\zeta(r)$ diverges more and more slowly. One might naïvely think the rate of divergence would gradually decrease to zero, after which there would be a region where the series converged. But this isn't what happens. Instead, after $r=1$ there is a drastic phase change in the behavior. The behavior jumps suddenly from divergence to convergence, without ever achieving a rate of divergence slower than $\log n$.
But $\log n$ is not very slow. Many series diverge more slowly. ($\sum\frac1p$ for prime $p$, for example.) If we imagine a space of possible behaviors of series, there is a largish region in this space that is avoided by the $\zeta$ series.
What explains the gap in possible behaviors of $\zeta$?
Is there a way to formalize my idea of a a space $\mathcal B$ of possible series behaviors, and a topology on that space? If so, is anything understood about maps from the space of sequences to $\mathcal B$ and what the range of such a map can look like?
Is there any way to turn this into less muddled question?
There are multiple notions to think about and in the end I suggest a way to think about your gap idea.
First, it is not clear why $\zeta(r)$ diverges more and more slowly. Divergence is a statement about non-existence of a limit. It's a yes/no and there is no in-between. You may be tempted to compare partial sums for equal $n$, but why do you choose equal $n$? That's an arbitrary (even though simplest) choice. You may as well compare the partial sums $S_r(\lfloor 1/(1-r)^n\rfloor)$ for different $r$. Therefore you take more terms for the "slower" series before you compare. It's a perfectly fine choice and shows that it's not easy to define a universal way of saying something is more divergent. Well, maybe the busy beaver function is more divergent as it diverges faster than any computable function.
At $r=1$ the series is still divergent and just because there is a simple expression $\ln x$ which is asymptotically only a constant away from the partial sum also does not have a special meaning. Maybe there are similar expression for $r<1$, maybe not... in any case according to the above argument it does not have a special meaning (unless you arbitrarily define how you choose to compare partial sums). There is no "gap" below $\ln n$ (it's all in the divergent range), but maybe above?
Now, here is a suggestion for how to understand what goes on between the divergent $\ln n+\gamma$ and the following convergent series for $r>1$. If you replace the summation by an integral, you can probably see that the sum is related to $\frac{1}{-r+1}n^{-r+1}$. For $r=1$ this would be $\frac10 n^0$ which for exactly zero fails to compute. Again, this is about existence and strictly yes or no. Indeed, the logarithm kind of behaves like the scaled zeroth power according to $\ln x=\lim_{\varepsilon\to 0} (x^\varepsilon-1)/\varepsilon$. So since the partial sums are related to $\frac{1}{s}n^s$ ($s=1-r$), there does not seem to be a gap between $s=0$ and $s>0$.
In that sense I don't really see a gap between $s=0$ and $s>0$, since the parameteric form of the partial sums is fixed to $\sim n^s$, and $\ln n$ "behaves" like $n^s$ with $s=0$.