Suppose $f\colon [a,b] \to \mathbb{R}$ is a continuous function with $f(a)<0$, $f(b)>0$. Can it be proved that there exists $s_1\leq s_2$ and $\epsilon>0$ such that $f(s)=0$ for all $s\in[s_1,s_2]$, whilst $f(s)<0$ for all $s\in [s_1-\epsilon, s_1)$ and $f(s)>0$ for all $s\in (s_2,s_2+\epsilon]$?
If not, what about in the case that one assumes $f$ is $C^1$, or smooth?

Counterexample
Let $f(x) = x^2 \sin^2(1/x) \operatorname{sign}(x)$ with $f(0)=0$ (plot). For $x<0$ the function oscillates between $0$ and negative values, and for $x>0$ it oscillates between $0$ and positive values. At $x=0$ the function crosses the $x$-axis, but it has zeros arbitrarily close to $x=0$ so there is no interval $(-\epsilon,0)$ on which $f$ is strictly negative, and no interval $(0, \epsilon)$ on which $f$ is strictly positive.