Proof: Let g be increasing on $[0,1]$ and continuous on $(0,1]$. Show that $f \in \mathbf{R}([a,b],g)$, where $f$ is defined on $[0,1]$ as follows. $f(x) = \frac{1}{k}$ if $x=\frac{1}{k}$ for some natural number $k$, and $0$ otherwise. In order to show that $f \in \mathbf{R}([a,b],g)$, it suffices to show that $f$ is continuous on $(0,1]$ (since $g$ is taken to be increasing on $[0,1])$. Then, let $\epsilon > 0$ be given, it is then our goal to find $\delta > 0$ such that $|x-y| < \delta$ implies $|f(x)-f(y)| < \epsilon, \forall \epsilon > 0$.
Case I: $x \ne \frac{1}{k}, k \in \mathbf{N}, k \ne 0$. Then, $|\frac{1}{x}-\frac{1}{y}| < \delta$ implies $|0|<\epsilon$. Hence, $f$ is continuous (In fact, $f$ is uniformly continuous as $\delta$ does not depend on $x,y \in (0,1]$).
Case II: $x = \frac{1}{k}, k \in \mathbf{N}, k \ne 0$. Then, $|\frac{1}{x}-\frac{1}{y}| < \delta$ implies $|\frac{1}{x}-\frac{1}{y}| < \epsilon$. Let $\epsilon > 0$ be given. Take $\delta = \frac{1}{\epsilon}$. Hence, $|\frac{1}{x}-\frac{1}{y}| < \delta=\frac{1}{\epsilon}$ implies $|\frac{1}{x}-\frac{1}{y}|<\frac{1}{\epsilon}< \epsilon$. Thus, $f$ is continuous on $(0,1]$ (in particular, $f$ is uniformly continuous). Thus, $f \in \mathbf{R}([a,b],g)$ since $g$ is increasing on $[0,1]$ and $f$ is (uniformly) continuous on $(0,1]$. QED.
You can calculate directly.
Fix $N>1$. Pick a partition $P=\{0=x_0<x_1<...<x_m=1\}$ with $mesh(P)<\delta<\frac{1}{N}$. Pick $c_i\in[x_i,x_{i+1}]$.Then $$S(f,P)=\sum_{i=0}^{m-1}f(c_i)(g(x_{i+1})-g(x_i)) =\sum_{i=0}^{j-1}f(c_i)(g(x_{i+1})-g(x_i)) +\sum_{i=j}^{m-1}f(c_i)(g(x_{i+1})-g(x_i)) $$ where $\frac{1}{N}\in [x_{j-1},x_j]$
Now, $g$ is uniformly continuous on $[\frac{1}{N},1]$, so for mesh size small enough, $|g(x_{i+1})-g(x_i)|<\epsilon$. At most $2N$ of the intervals which make the partition in $[\frac{1}{N},1]$ contain a non zero value of $f$. Further, $f$ is bounded by $1$ so we have, $$|\sum_{i=j}^{m-1}f(c_i)(g(x_{i+1})-g(x_i))|\leq 2N\epsilon.$$ Note the smaller we choose the mesh of the partition, the smaller $\epsilon$ will be so this term vanishes as the mesh size decreases.
To bound the first term, we may use $f(c_i)\leq \frac{1}{N}$ Then we have, $$|\sum_{i=1}^{j-1}f(c_i)(g(x_{i+1})-g(x_i))|\leq \frac{1}{N}(g(x_j)-g(0)) $$
Again, this term goes to zero as $N$ grows so the overall answer is $f$ is integrable and the integral is zero.