Is the tangent space to a critical submanifold a subspace of the kernel of the Hessian?

Question

Trying to solve a question I have been faced with another question.

Let $f:M\to\mathbb{R}$ be a smooth function and $b\in \mathbb{R}$ a critical value of it. Now the following relation is true? $$T_qf^{-1}(b)\subset \operatorname{Ker}\operatorname{Hess}_qf=E_0,$$ where $E_0$ is the eigenspace associated to the eigenvalue $0$.

FYI: The Hessian of $f$ at a critical point $q$ is a symmetric bilinear form $\operatorname{Hess} f_q$ s.t. $\forall v,w\in T_qM$, $$\operatorname{Hess} f_q(v,w)=V_q(W(f)),$$ where $V,W$ are the extensions of $v$ and $w$ to vector fields such that $V_q=v$ and $W_q=w$.

I would appreciate any comment

Answer 1

In the codimension-one case (where the level set is a hypersurface), you need to assume that $q$ is a critical point, not just that $b$ is a critical value. With this small change to your statement (which I assume you intended - it's necessary for the Hessian to be well-defined and thus for the claim to make sense at all), and a strong enough notion of submanifold (if you're allowing a self-intersecting immersion then $f(x,y) = xy$ is a counterexample), the result is true. Without loss of generality I will assume $b=0$ for simplicity.

Since we are assuming $f^{-1}\{0\}$ is a submanifold, we can choose coordinates $(x^1,\ldots,x^k,y^1,\ldots,y^{\ell}) : U \to B(0,1)$ on some neighbourhood $U \ni q$ so that $U \cap f^{-1}\{0\}=\{y=0\}$ and $q$ is the origin. In particular note that the $x^i$ restrict to give coordinates on the submanifold $\{y = 0\}.$ The fact that $q$ is a critical point gives us the extra information $\partial_{y_i} f(q) = 0$. Since Guiseppe's answer shows that $\partial_{x^i} \partial_{x^j} f(q) = 0$, it remains to show that $\partial_{x^i} \partial_{y^j} f(q) = 0$. The crucial fact we need to use is the following: not only is $f$ zero on $\{y = 0\}$, but it's nonzero everywhere else.

Case 1: $\ell = 1$, i.e. $f^{-1}\{0\}$ is a hypersurface and we have a single transverse coordinate $y=y^1$. It's not surprising this case is special, since it's the "expected" dimension (i.e. the dimension we would get from the regular value theorem if $0$ was a regular value). Since $f$ is a nonzero continuous function on the connected set $\{ y > 0 \}$, it has constant sign there; so combining this with the fact that it is zero on the boundary $\{ y = 0\}$ we conclude that $\partial_y f$ cannot change sign on $\{ y = 0\}$. Since $\partial_y f$ is zero at $q$, it is thus a smooth function with a local extremum at $q$, so it is critical there; i.e. $\partial_{x^i} \partial_y f = 0$ as desired.

Case 2. $\ell > 1$. In this case, the full complement $\{ y \ne 0 \}$ is connected; so $f$ has constant sign over $\{ y \ne 0 \}$ but is zero along $\{ y = 0\}$. Thus every point in $\{ y = 0 \}$ is a critical point of $f$, so $\partial_{y^j} f = 0 $ is constant on $\{ y = 0 \}$ and thus $\partial_{x^i} \partial_{y^j} f = 0$ as desired. Note that we didn't use the criticality assumption here - the function must be critical at this point to have such a small level set.

Answer 2

EDIT. There is an easy partial answer and a much more clever full answer, due to Anthony Carapetis.

---

Easy partial answer

Assuming that $q\in N\subset f^{-1}\{b\}$ and that $N$ is a smooth submanifold of $M$,

then

$\mathrm{Hess}_q(f)(v, w)=0$ for all $v, w\in T_q N$.

Why partial? Because this gives no information on $\mathrm{Hess}_q(f)(v, w)$ for $v\in T_qN$ and general $w\in T_qM$.

Proof. If $\gamma$ is a curve passing through $q$ (i.e. $\gamma(0)=q$) and contained in $N\subset f^{-1}\{b\}$, then $$f(\gamma(t))=b, \qquad \forall t, $$ and differentiating this relation twice we obtain $$\mathrm{Hess}_{\gamma(t)}\, f(\dot{\gamma}(t), \dot{\gamma}(t)) + df(\gamma(t)) \ddot{\gamma}(t)=0.$$ Evaluating at $t=0$, using that $df(\gamma(0))=0$, this yields $$\mathrm{Hess}_{q}\, f(\dot{\gamma}(0), \dot{\gamma}(0))=0.$$

Since $\gamma$ was an arbitrary curve on $N$, this last equation shows that $\mathrm{Hess}_q\, f(v, v)=0$ for all $v\in T_qN$.

This is enough to conclude, because $\mathrm{Hess}_q\, f$ is a symmetric operator. Indeed, if $H\colon V\times V\to \mathbb R$ is a symmetric bilinear operator on a real vector space $V$, then $H(v, v)=0$ for all $v\in V$ implies that $H(v, w)=0$ for all $v, w\in V$, because of the polarization identity.

---

Full answer

Assuming that there exists an open set $U\subset M$ such that $q\in U$ and $$N=U\cap f^{-1}\{b\}\quad \text{is a smooth submanifold of }M, $$

then

$\mathrm{Hess}_q(f)(v, w)=0$ for all $v\in T_q N$ and $w\in T_q M$.

Proof. This is Anthony's answer.

Remark. The main difference is in the result, which is stronger, because only one of the vectors is tangent to the level set $f^{-1}\{b\}$. The assumption is also slightly stronger, as it is assumed that the full level set has the structure of a submanifold (locally at least). In the easy case we merely assume that the level set contains a submanifold.

This is necessary to rule out the case of $f(x, y)=xy$. The origin of $\mathbb R^2$ is a critical point for $f$. The level set $f^{-1}\{0\}$ is not a submanifold in any neighborhood of the origin, due to self-intersection. We could consider the submanifold $N=\{(x, y)\in \mathbb R^2\ :\ y=0\}$, but then the result of the "Full answer" is false, as $$\mathrm{Hess}_{(0,0)} f( (1,0), (0,1) ) =1\ne 0, $$ even if $(1, 0)\in T_{(0,0)} N$.