I'm interested in knowing whether there's a "largest" function $f: [\delta, \infty) \to \mathbb{R_{>0}}$, where $\delta> 0$, such that $$\int_\delta^{\infty} \frac{f(x)}{x}\, dx < \infty$$ converges. ("Largest" here is in the sense of big-O asymptotics.) In particular, I'd like to know whether the smallest functions for which this integral diverges are the constant functions.
Obviously, any function for which the integral converges must satisfy $f(x) = o(1)$; that is to say, $\lim_{x \to \infty} f(x) = 0$. Any $f$ that satisfies $f = O(x^{-\epsilon})$ for some $\epsilon > 0$, conversely, obviously works. One can further construct large families of functions $f$ asymptotically between $1$ and any $x^{-\epsilon}$; that is, such that $f = o(1)$ but $x^{-\epsilon} = o(f)$ for every $\epsilon > 0$. (For example, pick two sequences of positive real numbers $a_n, b_n$ both tending to zero, and define $f(x) = \max a_n x^{-b_n}$.)
My one explicit attempt to construct a function $f$ along these lines, though, still gave a finite integral of $\frac{f(x)}{x}\, dx$. Define:
\begin{align*} f(x) = \begin{cases} x^{-1} & x \in [2, 4] \\ \frac{1}{2} x^{-1/2} & x \in [4, 16] \\ \frac{1}{4} x^{-1/4} & x \in [16, 256] \\ \frac{1}{8} x^{-1/8} & x \in [256, 65536] \\ \vdots & \vdots \\ 2^{-n} x^{-1/2^n} & x \in [2^{2^n}, 2^{2^{n+1}}] \\ \vdots & \vdots \end{cases} \end{align*}
Then \begin{align*} \int_2^\infty \frac{f(x)}{x}\, dx &= \sum_{n=0}^\infty \frac{1}{2^n} \int_{2^{2^n}}^{2^{2^{n+1}}} x^{-1-1/2^n}\,dx \\ &= -\sum_{n=0}^\infty \left. x^{-1/2^n}\right|_{x=2^{2^n}}^{x=2^{2^{n+1}}} \\ &= \sum_{n=0}^\infty \left( \frac{1}{2^{2^n - n}} - \frac{1}{2^{2^{n+1} - n}} \right) \\ &< \sum_{n=0}^\infty \frac{1}{2^{2^n - n}} \\ &< \sum_{n=0}^\infty \frac{1}{2^n} = 2. \end{align*}
Consider instead the function $g(x) = f(e^x)$. Evidently, $\lim_{x \to \infty} g(x) = \lim_{x \to \infty} f(e^x) = \lim_{x \to \infty} f(x) = 0$. Furthermore, we have $$\int_{\delta}^{\infty} \frac{f(x)}{x} \, dx = \int_{\ln \delta}^\infty g(x) \, dx$$ Then the question really becomes the following: does $\lim_{x \to \infty} g(x) = 0$ imply $\int_\delta^\infty g(x) \, dx < \infty$? We know this is false simply by considering $g(x) = \frac1x$. This would be equivalent to $f(x) = \frac1{\ln x}$.
If we aim to know the "largest" function whose integral converges, we can alternatively search for the "smallest" function whose integral diverges to gain some insight. Note that we can take any function that approaches $\infty$ as $x$ goes to $\infty$ and examine its derivative to arrive at a desirable "small" function, so consider, for example, the function $$h(x) = \ln \ln \ln \cdots \ln x$$ where $\ln$ is applied an arbitrary number of times. Regardless of the number of times we apply $\ln$, this function will still approach $\infty$, so we consider the derivative of this. We end up with $$f(x) = \frac{1}{\ln(x)\ln(\ln(x))\ln(\ln(\ln(x)))\cdots\ln(\ln(\cdots(\ln(x))\cdots))}$$ which satisfies the condition of your original problem. We can have a function approach $\infty$ arbitrarily slowly, which means that there's not really a "smallest" function with a divergent integral. We can perhaps say that $f(x)$ as just stated but with an infinite product is the "largest" function with a convergent integral, but this is a bit suspect.