Is this result true, and if so, how do I prove it?
Since the supremum of the lower sums of partitions of $[a, b]$ is zero, as $g$ is non negative, this means that for every $x , y$, $a \leq x < y \leq b$, $\inf_{t \in [x, y]} g = 0$.
Also, $g$ is continuous almost everywhere. Could we conclude that $g$ is zero at all its points of continuity?
If $g$ is Riemann integrable, it is Lesbesgue integrable. Because $\int_a^b g =0$, $g$ is zero almost everywhere.
As a non-negative map $f$ continuous at a point $a$ and positive at $a$ has a positive integral, $g$ is zero at all its points of continuity.