Suppose I have a continuously differentiable function $f:\mathbb{R}^N\rightarrow \mathbb{R}$ where the gradient is defined everywhere. Let $c$ be some constant in the range of $f$ and let $S=\{x \in \mathbb{R}^N: f(x)>c\}$. Assume $S$ is non-empty. Assume the gradient along the boundary of $S$, $\partial S$, is non-zero everywhere.
This is where I'm struggling a little with phrasing my question. If people have comments to help me fix this question's phrasing please tell me and I will change it ASAP. I have drawn the below picture to help show what I am asking.
I want to show that for any point, $p$, along the boundary of $S$, $\partial S$, if I draw a ball around $p$, the set $\partial S$ can only partition the ball into $2$ components.
On the left-hand side of the picture is an example where this condition can be seen to be the case for all $p$ on $\partial S$, whereas on the right-hand side it cannot be the case as the at the point $p$ labelled, the $\partial S$ divides the ball into 4 components.
Intuitively, I can explain why I think situations like on the right-hand side can't happen. The gradient $\nabla f$ must be normal to the level set and, in that example, I cannot draw a (non-zero) vector normal to $\partial S$ at $p$. However, I have no idea how to show this formally. Can anyone help me formalize my problem and solution?
Your intuition that the gradient needs to be nonzero is good. These kinds of ideas are formalized usually by the implicit function theorem or inverse function theorem. Here we will use the implicit function theorem, for which the idea is that "if $f$ is changing in a direction, then the solutions to $f=c$ locally are the graph of a function."
For a $C^1$ function $f: \mathbb{R}^N \to \mathbb{R}$ with $N > 1$, if the gradient at a point $p$ is nonzero, then some component is nonzero. Without loss of generality, the last component is nonzero. Then write $\mathbb{R}^N = \mathbb{R}^{N-1}\times \mathbb{R}$, with points written $(x, y)$ with $x \in \mathbb{R}^{N-1}$ and $y \in \mathbb{R}$. We have assumed that $\partial_y f(p) \neq 0$ and $f(p) = c$. Then the implicit function theorem tells us that there is an open ball $U \subseteq \mathbb{R}^{N-1}$ centered at $x_0$, where $(x_0, y_0) = p$, such that there is a unique $C^1$ function $g: U \to V \subseteq \mathbb{R}$ with $V$ a ball around $y_0$ such that $g(x_0) = y_0$ and $f(x, g(x)) = c$ for all $x \in U$ and the set in $U \times V$ where $f = c$ is the graph of $g$, i.e. $\{(x, g(x)): x \in U\} = \{(x,y) \in U \times V : f(x,y) = c\}$.
That is, locally, $g$ draws all the points where $f(x,y) = c$. In particular this rules out your second picture, where points of the boundary are approaching from a different direction. This nicely splits $U\times V$ in two, giving $A:=\{(x, y) \in U \times V : y > g(x)\}$ and $B:= \{(x, y) \in U \times V : y < g(x)\}$. At all the points in $A, B$ we know that $f(x,y) \neq c$. It is not difficult to see that $A,B$ are open and path-connected (open because $g$ is continuous, path-connected by using a vertical-horizontal-vertical path.) Thus $f-c$ cannot have a sign change in $A$ or $B$, so we must have $f(A) > c$ and $f(B) < c$ or $f(A) < c$ and $f(B) > c$ pointwise, so indeed the boundary is locally given by $g$ and locally splits $f$ into the two open sets $A, B$ where $f > c$ (inside $S$) and $f< c$ (outside $S$).