Let $M$ be a submanifold of $\Bbb{R^n}$ and $f:M\to \Bbb{R}$ continuously differentiable and such that $f^{-1}[a,b]$ has no critical points.
We define $\nabla f (a)$ by $\langle \nabla f(a),v \rangle=T_a f(v)$ for all $v\in T_a M$. Now for all $x\in f^{-1}([a,b])$ we have $\nabla f(x)\ne 0$.
How can I extend this in a neighborhood of $f^{-1}([a,b])$ ? It seems thatp I need bump function but I really don't "see" how can I do that.
Edit: sorry I forgot to say that $M$ is compact so does $f^1[a,b]$
I am not sure whether this is really what you are asking, but by construction $\nabla f$ is defined on all of $M$ and continuous. Hence $(\nabla f)^{-1}(\mathbb R^n\setminus\{0\})$ is open in $M$ and contains $f^{-1}([a,b])$ by construction. Hence $\nabla f$ is nonzero on an open neighborhood of $f^{-1}([a,b])$ in $M$.