After defining the Riemann integral over bounded subsets of $\mathbb{R}^n$, Munkres' Analysis on Manifolds defines the improper integral as follows:
Definition. Let $A$ be an open set in $R^n$; let $f : A \to R$ be a continuous function. If $f$ is non-negative on $A$, we define the (extended) integral of $f$ over $A$, denoted $\int_A f$, to be the supremum of the numbers $\int_D f$, as $D$ ranges over all compact rectifiable subsets of $A$, provided this supremum exists. In this case, we say that $f$ is integrable over $A$ (in the extended sense). More generally, if $f$ is an arbitrary continuous function on $A$, set $$f_+(x) = \max\{f(x), 0\}$$ and $$f_-(x) = \max\{-f(x) , 0\}.$$ We say that $f$ is integrable over $A$ (in the extended sense) if both $f_+$ and $f_-$ are; and in this case we set $$\int_A f = \int_A f_+ - \int_A f_-,$$ where $\int_A f$ denotes the extended integral throughout.
This allows one to compute integrals such as $\int_{-\infty}^\infty f$ or $\int_{\mathbb{R}^2} f$. Furthermore the extended integral also agrees with the original integral, so one can compute integrals over bounded subsets such as $\int_2^8 f$ or $\int_{[10,57]\times[3,5]} f$.
However, it seems that he does not give a way for us to compute common integrals like $\int_{[0, \infty)} f$, or $\int_2^\infty \int_3^\infty f$ for example. How are such unbounded non-open integrals defined? Does Munkres cover them anywhere, and if not, are there good resources that cover such (multidimensional) integrals rigorously?
To compute the integral $\int_{[0,\infty)} f$ using the above definition of an extended integral, remember that the stipulation that $f$ is non-negative means that if $A$ and $B$ are compact and $A \subset B$ then $\int_A f \leq \int_B f$ So the integral proceeds as it does in calculus, as $\lim_{t \rightarrow \infty}\int_{[0,t]} f$ since all of those sets are compact. Hope this helps.