The definition of Lebesgue outer measure is: $\forall A\in P(\mathbb{R}^n)$, $\mu^*(A)=inf\{\sum \mu(R_i)$, where the union of $R_i$ is a covering of $A$}. Then by the property of infimum, $\forall\epsilon>0$, there exists a sequence of $\{R_i\}$ covering $A$ such that $\mu^*(A)+\epsilon>\sum\mu(R_i)$, thus, $\sum\mu(R_i)-\mu^*(A)<\epsilon$.
Question: $\infty-\infty$ is not defined. What if both $\sum\mu(R_i)$ and $\mu^*(A)$ are $\infty$? Would the inequality above be wrong then?
When $\mu^{\ast}(A)=\infty$, then it means that every element in the infimum is of value $\infty$ (because the infimum cannot be empty, such a union always exists), so $\displaystyle\sum\mu(R_{i})=\infty$, for such $\epsilon$, just write $\mu^{\ast}(A)+\epsilon\geq\displaystyle\sum\mu(R_{i})$ since we define $\infty+a=\infty$ for any real number $a$.