Let $f: U \to \mathbb R$ be a $C^{\infty}$ function defined on a bounded open $U \subset \mathbb R^n$ with the property: \begin{align*} f(x) \to +\infty \text{ as } x \to \partial U. \end{align*} In $x \to \partial U$, I mean for every sequence $\{x_k\} \subset U$ such that $x_k \to y$ where $y \in \partial U$. It is not hard to show this property is equivalent to: for every $\alpha \in \mathbb R$, the sublevel set $S_{\alpha} = \{x \in U: f(x) \le \alpha\}$ is compact.
Now pick an arbitrary point $\zeta \in U$. Denote the sublevel set $S_{\zeta} = \{x \in U: f(x) \le f(\zeta)\}$ and assume it has nonempty interior. Now let $c = \sup\{\|\nabla^2 f(x)\|_2: x \in S_{\zeta}\}$. That is, the norm of Hessian of $f$ over $S_{\zeta}$ is bounded by $c$. We build a function $g: \mathbb R^n \to \mathbb R$ by \begin{align*} y \mapsto f(\zeta) + \langle \nabla f(\zeta), y-\zeta) + \frac{c}{2} \|y-\zeta\|_2^2. \end{align*} Then $g(\zeta) = f(\zeta)$ and $g$ majorizes $f$ on $S_{\zeta}$, i.e., $g(z) \ge f(z)$ for all $z \in S_{\zeta}$.
My question is whether the point $\tilde{\zeta} := \text{argmin} \{g(x): x \in \mathbb R^n\} = \zeta -\frac{1}{c} \nabla f(\zeta)$ is contained in $S_{\zeta}$.
This is not obvious to me since the inclusion of sublevel set is $S_{\zeta} \subset \{y: g(y) \le g(\zeta) = f(\zeta)\}$, not the other way. It is not necessarily $\tilde{\zeta}$ would lie in $S_{\zeta}$. My thought on this is: $g$ is continuous on $S_{\zeta}$ with $g(\partial S_{\zeta}) = f(\zeta)$ and nonconstant, the stationary point must lie in the inerior of $S_{\zeta}$. Is this a correct thinking?
After reading copper.hat's answer, I have this thought (possible wrong and will appreciate if someone points out): Denote the univariate function $\psi(t) = f(\zeta+ t(\tilde{\zeta}-\zeta)) = f(\zeta - \frac{1}{c} \nabla f(\zeta))$. Let $T$ be a connected component in $S_{\zeta}$ containing $\zeta$. The ray $\zeta + t(\tilde{\zeta} - \zeta) = \zeta - t\frac{1}{c} \nabla f(\zeta)$ will meet $T$ at another point (possibly more than one, we pick the first one of intersection), say $\xi$ with $\xi = \zeta + s(\tilde{\zeta} - \zeta)$. On the line segment $[\zeta, \xi]$, $\psi$ should dominate $f$. But we have $\psi(0) = f(\zeta)$ and $\psi(s) \ge f(\xi) = f(\zeta)$ since $\xi, \zeta \in \partial S_{\zeta}$. Should this imply the stationary point must lie in $S_{\zeta}$ even without the assumption on $U$ by Rolle's theorem and intermediate value theorem?
This answer assumes that $U$ is convex and that $S_\zeta $ is compact (so $c$ is finite).
If $\tilde{\zeta}=\zeta$, then we have $\tilde{\zeta}\in S_\zeta$, so we can assume that $\tilde{\zeta}\ne \zeta$.
I am also assuming that $\tilde \zeta \in U$.
Let $\gamma(t) = g(\zeta + t (\tilde{\zeta}-\zeta))$ and similarly $\phi(t) = f(\zeta + t (\tilde{\zeta}-\zeta))$. We have $\phi'(0) < 0$ and $\phi(0) = \gamma(0) = f(\zeta)$. Note that $\gamma$ is strictly decreasing (since $g$ is a convex quadratic and $\tilde \zeta$ is the minimiser).
Let $\Sigma = \{ t \in [0,1] | \phi(t) \le \phi(0) \}$. Note that if $[0,t] \subset \Sigma$ we have $\phi(t) \le \gamma(t)$.
Let $t^* = \sup \{ t | [0,t] \subset \Sigma \}$. Since $\phi'(0) <0$, we note that $\Sigma$ contains a set of the form $[0,\delta)$ for some $\delta >0$ and so $t^* >0$. Suppose $t^* <1$, then by continuity we have $\phi(t^*) \le \gamma(t^*) < \gamma(0) = \phi(0)$ and so there is some open interval $I \subset \Sigma$ containing $t^*$ which contradicts the maximality of $t^*$. Hence $t^* = 1$.