I know that the result is true for non-strict inequality:
If $f(x)$ and $g(x)$ are continuous, and $f(x)\leq g(x)$ when $x$ is a rational number, then $f(x)\leq g(x)$ $\forall x \in \mathbb{R}.$
However, I am not sure if it still holds when $$\leq$$ is replaced by $$<$$ throughout.
No, it is not true. Pick your favourite irrational number $\alpha$, and then let $$f(x)=-(x-\alpha)^2$$ $$g(x)=(x-\alpha)^2$$
(or just $f(x)=0$).