I am currently looking at the following theorem which I know how to prove from one side but not from the another side:
Theorem: If $a$ is a regular value of a smooth function $F:U\subset\mathbb{R}^3 \rightarrow \mathbb{R}$ and $p \in F^{-1}(a)=S$, then the tangent plane $T_pS=(\nabla F(p))^{\perp}$.
The definition of surfaces and tangent planes follow the textbook "Differential Geometry of Curves and Surfaces" by M. do Carmo.
Now, I know how to prove $T_pS\subset(\nabla F(p))^{\perp}$ simply by chain rule. But I have no idea how to prove the another side of the subset relation, because what I need to do is that given $u \in (\nabla F(p))^{\perp}$, I need a curve $\alpha:(-\epsilon,\epsilon)\rightarrow S$ such that $\alpha(0)=p$ and $\alpha'(0)=u$, but I have no idea how to find such a curve.
For $u\in ( \nabla F(p))^\perp$, $$ \frac{d}{dt} \ F(p+tu) =0$$
Here $c(t)=p+tu$ so that $$ F\circ c(t)= a + \frac{1}{2}t^2 D^2F(u,u) + O(t^3) $$
If $c_2(t)=p+tu+ \frac{1}{2}t^2 A \nabla F$, then $$ F\circ c_2(t)= a+ \frac{1}{2}t^2 \underbrace{\{ D^2F(u,u) + A \nabla F\cdot \nabla F \}}_{=K} + O(t^3) $$
For some $A$, we have $K=0$ so that we can approximate curve $\alpha$ with $F|\alpha =a$.