Background: To be more general, if an improper integral of the form (where $f $ is unbounded at $0$)$$\int_0^bf(x)dx:=\lim_{\epsilon\rightarrow0^+}\int_\epsilon^bf(x)dx$$ converges, then can I conclude that for $x\in[0,b]$, the function $$g(x)=\int_0^xf(t)dt$$ is contiunuous?
Attempt: I guess this is true because its analogue is true: if f is Riemann integrable over $[0,b]$, then $\int_0^xf(t)dt $ is a continuous function of $x$ for $x\in[0,b]$.
Why would I care: Because I want to use a claim: since $$g(0)=\int_0^0f(t)dt=\lim_{\epsilon\rightarrow0^+}\int_\epsilon^\epsilon f(x)dx=0$$, then by the proposed continuity of $g$, I can have $$\lim_{x\rightarrow0^+}g(x)=\lim_{x\rightarrow0^+}\int_0^xf(t)dt=g(0)=0.$$
Assuming that $f$ is Riemann integrable on every interval of the form $[\epsilon, b] $ we can see that $$h(x) =\int_{x} ^{b} f(t) \, dt$$ is continuous on $(0,b]$ and further we are given that limit $\lim_{x\to 0^{+}}h(x)=L$ exists. Defining $h(0)=L$ we can now see that by definition $h$ is continuous on $[0,b]$.
Your function $g(x) $ is related to $h(x) $ via $g(x) +h(x) =L$ and hence $g$ is also continuous on $[0,b]$.