This question was left as an optional exercise in my lecture notes and says the argument uses compactness. I came up with an argument which doesn't require compactness and I wondering if it is correct.
Let $F=\cup_{n=1}^{\infty}\lbrack a_n,b_n\rbrack$
For each n,
$\lbrack a_n,b_n\rbrack \subseteq O_n$, where $O_n=(a_n-\frac{\epsilon}{2},a_n) \cup(a_n,b_n) \cup(b_n,b_n+\frac{\epsilon}{2})$
As $\lbrack a_n,b_n\rbrack$ is defined as $(a_n,b_n)\cup\{a_n,b_n\}$ this implies
$(O_n\backslash\lbrack a_n,b_n\rbrack) =(a_n-\frac{\epsilon}{2},a_n) \cup(b_n,b_n+\frac{\epsilon}{2}) \backslash\{a_n,b_n\}$
Thus,
$m_{*}(O_n\backslash\lbrack a_n,b_n\rbrack) \leq m_{*}(a_n-\frac{\epsilon}{2},a_n) + m_{*}(b_n,b_n+\frac{\epsilon}{2}) - m_{*}\{a_n\} - m_{*}\{b_n\} =\epsilon$
By the subadditivity of the lebesgue measure and as countable sets having measure $0$.
Thus $\lbrack a_n,b_n\rbrack$ is measurable. As the countable union of measurable sets is measurable $F$ is measurable.