My lecture notes on the gradient state the following:
For $f : U \to \mathbb{R}$ differentiable consider the level set
$N_w = \{v \in U : f(v)=w\}$ where $U \in \mathbb{R}^n$.
Suppose that $c : I \to N_w \in U$ is a differentiable curve. Then $f \circ c = w$ and so
$0=\frac{d}{dt}(f \circ c)=\langle grad f(c(t)),c'(t)\rangle \iff grad (c(t)) \perp c'(t)$.
Since this holds for any differentiable curve running through $N_w$, we can say that the gradient vector of $f$ is perpendicular to the level sets $N_w$.
The proof is clear to me, but this brings up the question whether such a curve $c(t)$ exists. I've read a couple of other posts about this and it seems that the existence of such a function is guaranteed by the implicit function theorem.
Implicit function theorem (IFT):
Let $\Omega \subset \mathbb{R}^n \times \mathbb{R}^k = \mathbb{R}^{n+k}$ be open and $f : \Omega \to \mathbb{R}^k, (x,y) \to f(x,y)$ be continuously differentiable. Moreover, let $(a,b) \in \Omega$ be a point with $f(a,b) = 0$, such that
\begin{equation*} J^Y_f = \begin{pmatrix} \frac{\partial f_1}{\partial y_1} & \cdots & \frac{\partial f_1}{\partial y_k} \\ \vdots & \ddots & \vdots \\ \frac{\partial f_k}{\partial y_1} & \cdots & \frac{\partial f_k}{\partial y_k} \end{pmatrix} \end{equation*}
satisfies the following equivalent conditions at $(a,b)$:
$(*) J^Y_f(a,b)$ is invertible $\iff$ rank $J^Y_f(a,b) = k \iff det J^Y_f(a,b) \neq 0$.
Then there are neighbourhoods $X \subset \mathbb{R}^n$ of $a$ and $Y \subset \mathbb{R}^k$ of $b$ with $X \times Y \subset \Omega$ as well as a continuously differentiable mapping $g : X \to Y$ with
$f(x,y)=0$ for $(x,y) \in X \times Y \iff y=g(x)$ for $x \in X$.
Now the theorem gives a way to express some of the variables as a function of the others, but for the statement about the gradient we would need a curve $c(t)$ parameterized by $t$ where $c(t)$ is a vector in $\mathbb{R}^n$.
How can we parameterize the graph $(x,g(x))$ by $t$?
Thanks a lot!
Once you know your level set is (locally, near $p$) a graph over some coordinate plane, just take a line in that plane passing through the point $\bar p$ corresponding to $p$ and parametrize that line by $t$. That is, you get $x=x(t)$ a parametrized line with $x(0)=\bar p$, and then $c(t) = \big(x(t),g(x(t))\big)$.