Though a large class of functions are Lebesgue integrable, for certain class of functions, say for example $f(x)=1/x$ Lebesgue integral over the interval $[-a,a]$ for $a>0$ is undefined. But the limit $$\lim\limits_{\epsilon \rightarrow 0^+} \int_{[-a,a]\setminus(-\epsilon,\epsilon)} \frac1x dx=0$$ Thus, in some sense improper Lebesgue integrals (as considered above) of certain functions can be finite even when there Lebesgue integral (under the usual definition) is infinite.
Is this the motivation behind considering the limit as $\epsilon \rightarrow 0$ while defining principle value distribution rather that simply considering the Legegue integral with $\epsilon=0$?