To prove that absolute continuity implies $\epsilon-\delta$ criterion, it is said that finiteness is required. The proof involves construction of sequences of measurable sets, where their upper bound and lower bound are implied by the construction itself. I don’t really see explicitly where the proof utilises finiteness. I do know that the finiteness is required due to a counterexample given in the text.
Edit: The proof that I mentioned is the one in the link titled Step 1. https://www.math3ma.com/blog/absolute-continuity-part-two