The polynomial family of curves $y=x^n$ all have a common quality to their shape, their growth rate (e.g. considering big-O and little-o notation in complexity theory) and we all know that when $n\ne -1$ then $\int x^n \mathrm{d}x$ is another polynomial, and we also know that $\int x^{-1}\mathrm{d}x$ is not another polynomial, but the logarithm, which is qualitatively different when considering shape, growth rate, etc.
What bugs me, put verbally and non-rigorously is this: we have a situation that for all polynomials except one, integrating that polynomial yields another polynomial, but this special case discontinuously changes the nature of the curve.
Considering $\int x^{-1+\epsilon}\mathrm{d}x$ as $\epsilon \to 0$, it seems that for any infinitesimal-but-nonzero $\epsilon$ the curve before and after integration has all of the characteristics of being polynomial, so how do we describe the qualities of the curve changing as we approach the limit?
It seems that, by choosing suitably small $\epsilon$ I can construct a single-term polynomial that approximates the logarithm to an arbitrary degree of accuracy, but that sounds absurd. (Maybe my intuition is just wrong.)
Is there a rigorous characterization of how we transition from polynomials to logarithms in this way, and does it say anything interesting? Or is it just that my gut is wrong and what seems absurd to me is actually trivial?
Years ago I was puzzled by what I think is the same issue: The discontinuity of $\int x^n$ as a function of $n$ at $n=-1$. Then I realized
"Advanced" explanation: The problem goes away when you realize that $\int f$ is really not a function, rather it's an equivalence class of functions modulo constants. If you regard $\int$ as a mapping from, say, $C((0,\infty))$ to $C^1((0,\infty))/\Bbb R$ then $\int x^n$ is continuous at $n=-1$.
More elementary version of more or less the same explanation: The problem goes away if you consider the "$+c$", choosing it properly. Define $$I_n(x)=\frac1{n+1}(x^{n+1}-1)\quad(n\ne-1).$$Then $$\lim_{n\to-1}I_n(x)=\log(x)\quad(x>0).$$
Exercise: Figure out how this answer is really the same as metamorphy's commment...