This is a question from my analysis textbook.
Suppose $f$ is differentiable at the point $c$, and that $s$ is a function continuous at $0$ and $s(h)=0$ only at $h=0$. Prove using the $\varepsilon - \delta$ definition of the limit that
$$\lim_{h \to 0} \frac{f(c+s(h))-f(c)}{s(h)} = f^{\prime} (c)$$
My attempt: Let $\varepsilon > 0$ be given. Since $f$ is differentiable at $c$, there exists $\delta ^{\prime} >0 $ such that if $|y|< \delta^{\prime}$ then we have $$\left| \frac{f(c+y)- f(c)}{y} - f^{\prime} (c) \right| < \varepsilon$$
Now since $s$ is continuous at $0$, there exists $\delta > 0$ such that if $|h|<\delta $ then $|s(h)|<\delta^{\prime}$.
Hence, there is a $\delta >0$ such that if $|h|<\delta$, then we have
$$\left| \frac{f(c+s(h))- f(c)}{s(h)} - f^{\prime} (c) \right| < \varepsilon$$ as desired .
Is my proof correct? The only thing that bothers me here is the hypothesis that "$s(h)=0$ only at $h=0$". What would go wrong if $s$ was zero at some other point?
You've slightly misstated the definition of a limit. The meaning of $\lim_{y \to 0} g(y) = L$ is that for any $\epsilon > 0$ there exists $\delta > 0$ with the property that $$ \color{red}{\boldsymbol{0}} < |y| < \delta \implies |g(y) - L| < \epsilon.$$
Thus for $f$ to be differentiable at $c$ it must be that for every $\epsilon > 0$ there exists $\delta' > 0$ with the property that $$ \color{red}{\boldsymbol{0}} < |y| < \delta' \implies \left| \frac{f(c+y) - f(c)}{y} - f'(c) \right| < \epsilon.$$
Since $s$ is continuous at $0$ and $s(0) = 0$, there exists $\delta > 0$ with the property that $$|h| < \delta \implies |s(h)| < \delta'.$$ Since $s(h) = 0$ only when $h = 0$, $|h| > 0$ implies $|s(h)| > 0$ too. Thus $$\color{red}{\boldsymbol{0}} < |h| < \delta \implies \color{red}{\boldsymbol{0}} < |s(h)| < \delta'.$$ Of course this gives you $$\color{red}{\boldsymbol{0}} < |h| < \delta \implies \left| \frac{f(c+s(h)) - f(c)}{s(h)} - f'(c) \right| < \epsilon$$ which is the definition of the statement $$\lim_{h \to 0} \frac{f(c+s(h)) - f(c)}{s(h)} = f'(c).$$