I am reading about barrier functions in control engineering/dynamical systems. These tools are used to prove that the system is forward invariant with respect to a set $\mathcal{C}$ (i.e., starting in $\mathcal{C}$, we stay in $\mathcal{C}$ at all times). The concept is similar to the one of Lyapunov functions for stability. Here is the main idea:
Let the dynamical system be $\dot{x} = f(x)$. For simplicity, $x \in \mathbb{R}$. Let also $\mathcal{C}$ be defined as:
$\mathcal{C} = ${$x \in \mathbb{R}^n: h(x) \geq 0$}
$\partial\mathcal{C} = ${$x \in \mathbb{R}^n: h(x) = 0$}
$\text{Int}({\mathcal{C}}) = ${$x \in \mathbb{R}^n: h(x) < 0$}
Where $h: \mathbb{R}^n \to \mathbb{R}$ is a smooth function.
We can see that if $h(x(t)) \geq 0$ for all $t \geq t_0$, then the system is forward invariant with respect to $\mathcal{C}$.
Using Nagumo's theorem (see Theorem 3.1 of [1]), the condition that $h \geq 0$ is fulfilled if
$$ \dot{h} \geq 0 \text{ whenever } h = 0 \qquad \text{(1)} $$
Clearly this would translate directly to:
$$ \dot{h}(x) \geq 0 \quad \forall x \in \partial \mathcal{C} \qquad \text{(2)} $$
However, it seems that to make this step you need to impose the condition that $\frac{\partial h}{\partial x} \neq 0$ for $x \in \partial \mathcal{C}$ (see, for instance, the first paragraph in section 1.A of [2]). I do not know where this condition comes from.
I know it makes sense: if $h(x)$ has an inflection point at $\partial \mathcal{C}$, then $\dot{h} = \frac{\partial h}{\partial x} \frac{\partial x}{\partial t} = 0$, and (1) is fulfilled no matter the value of $\frac{\partial x}{\partial t}$ (even if $x$ is "leaving" the set $\mathcal{C}$). I just don't know how it appeared when going from (1) to (2).
[1] Blanchini, F., Set invariance in control, Automatica 35, No. 11, 1747-1767 (1999). ZBL0935.93005.
[2] Ames, Aaron D.; Xu, Xiangru; Grizzle, Jessy W.; Tabuada, Paulo, Control barrier function based quadratic programs for safety critical systems, IEEE Trans. Autom. Control 62, No. 8, 3861-3876 (2017). ZBL1373.90092.
From (1) to (2) you do not need the non-singularity of $h$. It is straight from definitions that $\dot h \geq 0$ on $\partial C$.
To use Nagumo's Theorem, there is one fact we must establish: that for all points $x\in\partial C$ we have that the vector field $f(x(t))$ lay in what is known as the tangent cone of $S$ at $x$. That condition essentially means that $\dot x$ is somehow pointing "into" the set $C$ instead of "out of". The definition is a bit opaque at first, and ideally we would not want to establish that $f(x(t))$ lies in the tangent cone explicitly. So how can side-step this? There is a comment on [pg. 1751, 1] saying,
Blanchini says here that when $\partial C$ is smooth, really all we need to be sure of is that $f(x(t))$ lies on "inner side" of the tangent half-space. How does having a non-singular $dh$ help with this? Well, fixing $x \in \partial C,$ we can use an appropriate level set theorem (see [Corollary 5.14, 3]) to argue that $h$ defines $\partial C$ as a regular level set locally and thus conclude that $\partial C$ is a smooth embedded submanifold of the ambient space. That isn't all, however. Notably, being an embedded submanifold ensures there is a slice coordinate chart locally around $x$ where $\partial C$ is flat and $dh$ points orthogonal to the flattened $\partial C$ in the new coordinate system. That is, there exists a local diffeomorphism $z = \Phi(x)$ on an open set $U$ where,
$$ U \cap \partial C = \{ (z^1, \ldots, z^{n-1}, 0) \colon z \in U\} $$
where $dh = dz^n$. We can use this to establish that $f$ is inside the tangent cone. Observe that on $\partial C$,
$$ \dot h(x(t)) \geq 0 \iff dh(f) \geq 0 $$
viewing $dh$ as a form and $f$ as a vector field. In the now flattened $\partial C$, $dh = dz^n$, so really we have,
$$ dz^{n}(f) \geq 0 $$
From this, deduce that $f$ lies in the tangent halfspace coinciding with the tangent cone.
The non-singularity of $dh$ is thus used to simplify the evaluation of the tangent cone criterion by saying that $h$ actually defines $\partial C$ as smooth embedded submanifold and leveraging this easy to evaluate condition for the tangent cone. That is not to say that you couldn't use Nagumo's Theorem without non-singularity, but that you'd have to painstakingly evaluate that tangent cone criterion at those singular points.
[3] J. M. Lee. Introduction to Smooth Manifolds. 2nd Ed.