Let $f: \ [a, b]\rightarrow \mathbb{R} $ be a continuous function. Since it is continuous, it is Riemann-integrable.
Let $F: \ [a, b]\rightarrow \mathbb{R}$ be given by $ F(x)=\int_a^x f(t)dt $. By the fundamental theorem of calculus, on the open interval $(a, b), $ $ F'(x)=f(x) $ (i.e. $F$ is an antiderivative of $f$).
Suppose that we want to calculate the integral $\int_a^b f(x)dx$, i.e. the value of $F(b)$. In practice we do that by finding any antiderivative of $f$ and by computing its values at $x=a$ and $x=b$. But, if we are being pedantic, we can't really do that because the fundamental theorem says that $F'(x)=f(x)$ only on the open interval! $F$ is defined on a closed interval so it is not differentiable at points $x=a$ and $x=b$ (at least not by the common definition of derivative). The fundamental theorem does not say anything about the values of $F$ at the boundary points.
So how could this 'formal error' be fixed? How can we define the values of $F$ at $x=a$ and $x=b$ rigorously?
One way I could think of doing this is that we define $F(a)=0$. This is intuitive but could it also be according to some axiom of integrals? Then I would also want to define $F(b)=\lim_{x \rightarrow b-} F(x)$ but I'm not sure how to justify it and how to guarantee that the limit exists.