My textbook's proof that the Lagrange multiplier method is valid begins:
Let $X(t)$ be a differentiable curve on the surface $S$ passing through $P$
Where $S$ is the level surface defining the constraint, and $P$ is an extremum of the function that we're seeking to optimize. But how do we know that such a curve exists?
$S$ is specifically defined as the set of points in the (open) domain of the continuously differentiable function $g$ with $g(X) = 0$ but $\operatorname{grad}g(X)\ne0$. The function $f$ that we're seeking to optimize is assumed to be continuously differentiable and defined on the same open domain as $g$, and $P$ is an extremum of $f$ on $S$.
By the Implicit Function Theorem, near $P$ you can represent your level surface as a graph, say $z=\phi(x,y)$, where $\phi$ is continuously differentiable. If $P=\phi(a,b)$, take any line through $(a,b)$ and you get a nice curve.