I know there are functions which are Riemann integrable but not Lebesgue integrable, for instance, $$\int_{\mathbb{R}} \frac{\sin(x)}{x} \mathrm{d}x$$
Is Riemann integrable and it is easily shown that its value is $\pi $, nevertheless, it is not Lebesgue integrable because $$\int_{\mathbb{R}} \left|\frac{\sin(x)}{x}\right| \mathrm{d}x$$ diverges. What is going on here? I thought Lebesgue integration was supposed to be an extension of Riemann integration, kind of like principal values are an "extension" of the notion of improper integral, and if an improper integral converges, it is equal to its principal value, why then every function that is Riemann integrable is not Lebesgue integrable?
Of course the opposite is also true, there are functions which are Lebesgue integrable but not Riemann integrable, so what is going on here? How does one integral notion extend the other and how do they not contradict themselves?
Thank you.
The following is a bit of a ramble, but I hope you find it a useful collection of information.
The Riemann integral is only defined for bounded functions on bounded intervals, which are all Lebesgue-integrable. It's the extension to the improper Riemann integral that can integrate functions that are not Lebesgue-integrable.
We recall that a function $f$ is improperly Riemann-integrable on $(a,b)$ if $\int_c^d f$ exists for all $c,d$ with $a<c<d<b$ and $\lim_{\substack{c \to a \\ d \to b }}\int_c^d f$
An easy way to understand the difference is that if $f$ is Lebesgue-integrable, $\lvert f \rvert$ must also be Lebesgue-integrable (basically because this is how we define the Lebesgue integral for nonpositive functions). $\sin{x}/x$ does not satisfy this, so cannot be Lebesgue-integrable.
Another way to understand the Lebesgue integral is via the Daniell construction: we take a vector space of basic functions $\mathcal{F}$ (so that in addition $f \in \mathcal{F}$ if and only if $|f| \in \mathcal{F}$) and a map $I: \mathcal{F} \to \mathbb{R}$ that satisfies linearity, positivity ($f \geq 0 \implies I(f) \geq 0$) and continuity (if $f_n$ is a nonincreasing sequence converging pointwise to $0$, then $I(f_n) \to 0$). One can then define an integral on functions representable as a monotone limit of the functions in $\mathcal{F}$, with some subtleties to deal with negative functions. The advantage here is that one may choose the basic integral $I$ as the Riemann integral and the elementary functions as the continuous functions on a finite interval, or compactly supported continuous functions, and then this construction can be proven to give Lebesgue's integral.
The usefulness of the Lebesgue integral does not really lie in extending the Riemann integral unilaterally. For this the Gauge/Henstock–Kurzweil integral is a much better idea, and indeed, functions like the characteristic function of the rationals are not common in the applications of integration for which the Lebesgue theory is favoured (that's most of them). Instead, we have found that it has other advantages:
The above are the modern reasons for using the Lebesgue integral: this can hardly have been Lebesgue's motivation, which was likely rather more rooted in the extension you suggest. It's also very difficult to exhibit a non–locally-Lebesgue-integrable function on a finite interval: one needs a lot of the axiom of choice.
We've essentially found there are two common sorts of integral one can define on the real numbers: "absolute" integrals, that tend to be the Lebesgue integral if they are sufficiently general (In this category we have Daniell, Mikusiński and McShane, for example). The other sort are the more general non-absolute ones, such as the gauge integral.