A bounded function which is Riemann integrable on any interval $[a+\epsilon, b]$, $\epsilon \in (0,b-a)$

Question

Let $f:[a,b] \to \mathbb{R}$ be a bounded function which is Riemann integrable on any interval $[a+\epsilon, b]$, $\epsilon \in (0,b-a)$. Prove that $f$ is Riemann integrable on $[a,b]$.
I wanted to define the functions $F_\epsilon:[a+\epsilon, b]\to \mathbb{R}$, $F_\epsilon(x)= \int_x^b f(t) dt$. These functions are continuous because $f$ is Riemann integrable on all these intervals.
We have that $F_\epsilon(a+\epsilon)=\int_{a+\epsilon}^{b}f(t)dt$ and I thought that if we took the limit here as $\epsilon \searrow0$ we would get that $\int_a^b f(t)dt$ exists, but I don't think that this reasoning really works.
I would like to know what you think about this and how you would solve it if my way doesn't work (I am inclined to believe that it doesn't).
EDIT: As pointed out in the comments below, my solution should work if we use some facts about Lebesgue integration. As I do not really know too much about this, I would like you to tell me what exactly is needed to make my solution rigorous.

Answer 1

Suppose $|f|\le M$ on $[a,b].$ Let $0<\epsilon <b-a.$ Then $f$ is Riemann integrable on $[a+\epsilon,b],$ so there is a partition $P$ of $[a+\epsilon,b]$ such that

$$U(f,P)-L(f,P) <\epsilon.$$

Let $Q= \{a\} \cup P.$ Then $Q$ is a partition of $[a,b]$ and

$$U(f,Q)-L(f,Q)$$ $$= (\sup_{[a,a+\epsilon]} f-\inf_{[a,a+\epsilon]} f)\cdot \epsilon + U(f,P)-L(f,P)$$ $$ < 2M\epsilon+\epsilon.$$

This proves $f$ is Riemann integrable on $[a,b].$