Let $S$ be a surface in $\mathbb{R}^3$ defined by the graph $z=f(x,y)$ of a smooth function $f: \mathbb{R}^2 \to \mathbb{R}$. Suppose furthermore that $S$ satisfies the second-derivative test for a local max or min at a critical point $p^* = (x^*,y^*)$; i.e. the Hessian matrix $H(x,y)$ of $f$ satisfies
det$(H)>0$, and either $H_{xx}>0$ or $H_{xx}<0$
when evaluated at $p^*$.
I'm looking for a proof (or counterexample) of the following statement:
Suppose $f(x^*,y^*)=c^*$. Then horizontal slices $f(x,y) = c$ include closed curves for values of $c$ sufficiently close to $c^*$ on either the right or the left.
This is obvious for elliptic paraboloids $f(x,y) = x^2/a^2+y^2/b^2$. Otherwise we can Taylor expand around the critical point so that the nontrivial leading-order terms are quadratic in $x$ and $y$, but it's not clear how to show that the intersections contain 'just slightly perturbed ellipses near $p^*$' in that case.
So here is a sketch.
For simplicity, let us consider the case where $f:\mathbb{R}^2\to\mathbb{R}$ attains a local minimum at $p=(0,0)$ with $f(p)=0$. By virtue of the inverse function theorem (for $Df$), $0$ is an isolated critical point of $f$, and so, we may restrict $f$ to a small ball around $0$ and assume that $0$ is the only critical point. Restricting the domain of $f$ further, we may also assume that there exist $\delta>0$ and $\epsilon>0$ such that for any $q$ in the domain with $|q|>\delta$ we have $f(q)>\epsilon$ (this can be shown, for example, by Taylor's theorem).
Now, let $0<a<\epsilon$ lie in the image of $f$. Then $a$ is a regular value, and so, the level set $f^{-1}(a)$ is a regular curve. On the other hand, this level set is closed (as a subset of $\mathbb{R}^2$) and contained in the closed $\delta$-ball around zero and hence compact.