Suppose I can bound the running time of my algorithm as $O(a_N N^2)$ for any positive increasing sequence $\{a_N\}$ that diverges to infinity. Does this imply that my algorithm's running time is actually $O(N^2)$?
N.B. I understand that the running time can be bounded by $O(N^{2+\varepsilon})$ for any $\varepsilon > 0$, and by $O(N^2 \log\log(N))$. I would like to understand if the big O notation has "this sort of continuity property".
Yes it does. Let $t_N$ denote the algorithm time. Let $f_N = \max\{t_M/M^2:M\le N\}$. Suppose $f_N$ diverges. Then $t_N$ fails to be $O(\sqrt{f_N} N^2)$. Therefore $f_N$ does not diverge. Since $f_N$ is a non-decreasing sequence, it must be bounded. Set $C= \sup f_N$. Then $t_N \le C N^2$.