In John B. Conway's book he defines Riemann integrability, and then defines improper integrals based off of that. He says $f:(a,b)\to\mathbb{R}$ is improperly integrable if it is integrable on all $[c,d]\subseteq(a,b)$ and $$\lim_{c\to a^+}\lim_{d\to b^-}\int_c^df$$ exists and is finite.
Without defining any other form of integrability he claims that if $f:(a,b)\to\mathbb{R}$ is absolutely integrable (that is $|f|$ is integrable), then it is integrable. Consider
$$f:=\begin{cases}1&x\in\mathbb{Q}\\-1&x\not\in\mathbb{Q}\end{cases}.$$
This modified Dirichlet function is absolutely integrable on any finite $(a,b)$ for $|f(x)|\equiv 1$. Conway's claim asserts that $f$ is then integrable. If $[c,d]\subseteq(a,b)$, then would not $U(f|_{[c,d]})=(d-c)$ and $L(f_{[c,d]})=-(d-c)$? In which case $f$ is not Riemann Integrable on $[c,d]$, hence not improperly integrable on $(a,b)$.
In fact, if $|f|$ is improperly integrable in the above sense, then $f$ isn't necessarily integrable on compact sets $[c,d] \subset (a,b)$. You have already mentioned the typical example. Probably, Conway had in mind that $f$ is continuous on $(a,b)$, or piecewise continuous. Then $f$ is improperly integrable, if $|f|$ is improperly integrable.
Note that you can replace $\mathbb{Q}$ by a non-measurable set $E$, say a Vitali-set, to get a function with is non-measurbale, but |f| is measurable.