Many fields in mathematics start from the "dirty" approach. In calculus we do all sort of $\epsilon$-$\delta$ stuffs, until topology gives an elegant formulation using open sets. A first course in linear algebra usually starts with defining matrices, their multiplications, determinants, etc, then the whole theory is built upon that. But if we start from vector spaces and linear operators on them, everything seems to fit in much more cleanly.
What about integrals? There are at least two approaches to define Riemann integrals: upper and lower integrals; or Riemann sums of partitions. None of them looks elegant to me. So, is there a "clean" way to develop the theory of integrals?
Let me say first that I fully agree with the comments posted above.
In any case, the answer to what you want depends heavily in what you mean by "clean" and, more importantly, what you mean by "theory of integrals".
Regarding "clean", your example is open sets as opposed to $\varepsilon$-$\delta$. But when you want to do concrete manipulations with open sets in a metric space you end up dealing with balls, and you prove things exactly by using $\varepsilon$-$\delta$; so it's not "cleaner" at all. Just more general, as Arturo said.
Regarding the "theory of integrals": if you want to obtain an integral that represents what the Riemann integral represents (i.e. "the area below the curve") you will have to deal with some kind of limits of sums. I personally prefer Lebesgue's approach, which is cleaner and more general, but it still implies a fair amount of "dirty".
If you dispense with the need of your integral being "the area below the curve", you might notice that all an integral does is to assign a number to a function in a linear way. So you might think of a locally compact Hausdorff space $X$, consider $C_c(X)$, i.e. the complex-valued compactly supported continuous functions on $X$, and think of all the functionals (i.e. scalar valued, linear functions) $$ \varphi:C_c(X)\to\mathbb{C}. $$ This space is well-understood, and every such functional is an "integral" in some proper sense (the Riemann integral being just a very particular example in the case where $X$ is a closed hypercube in $\mathbb{R}^n$). This last point of view is a lot "cleaner" to my taste, but it is probably not what you are looking for.