On the hessian matrix and relative minima

Question

I'm asked to prove the following statement:

Let $\mathbb{A} \subseteq \mathbb{R}^n$ an open subset and $f: \mathbb{A} \to \mathbb{R} / f \in \mathbb{C}^3 \wedge (P \in \mathbb{A} : \nabla f(P)=0)$. Suppose the hessian matrix of f is defined positive, prove that P is a strict relative minima of f.

A similar proof has already been posted here, however, there is a difference and I'd like to know whether my approach is right. Approximating $f$ by its Taylor's polynomial of grade 2 centered on P, we get the following expression:

$$f(x) = f(P) + \frac{1}{2} \langle H_f(P) \cdot (x-P), x-P \rangle + R(P) \cdot (x-P)$$

where $R(P) \cdot (x-P) = \frac{1}{6}\langle D^3_f(c) \cdot (x-P), x-P\rangle$, c being some point between x and p. By replacing $x$ with $P + tV$, $t \in \mathbb{R} \wedge V \in \mathbb{A}$, we get:

$$f(P + tV) = f(P) + \frac{1}{2} \langle H_f(P) \cdot (tV), tV \rangle + R(P) \cdot (tV)$$

Thus, by taking $t$ as a common factor from the vector product in the second term, we get:

$$f(P + tV) = f(P) + \frac{t^2}{2} \langle H_f(P) \cdot V, V \rangle + R(P) \cdot (tV)$$

By definition, P is a strict local minima iif $f(P) < f(P + tV)$ when $tV$ is sufficiently close to 0, hence replacing the equation we get:

$$0 < \frac{t^2}{2} \langle H_f(P) \cdot V, V \rangle + R(P) \cdot (tV)$$

And since $\langle H_f(P) \cdot V, V \rangle$ is the quadratic form associated to $H_f(P)$, which by hypothesis is defined positive (therefore being higher than 0), we know that the first term is higher than 0.

It is the second term I'm having trouble with: how can I prove that $R(P) \cdot (tV) \ll \frac{t^2}{2} \langle H_f(P) \cdot V, V \rangle$ and thus it can't make the statement false?. I'm also worried about how rigorous this proof is, since I'm not entirely sure that approaching the problem with the $x = P + tV$ replacement is entirely correct.

Answer 1

I'm not entirely sure that approaching the problem with the $x = P + tV$ replacement is entirely correct.

Indeed, the reasoning is insufficient in the sense that a function $f$ can admit a smooth curve of points $Q$, passing through $P$ and with $P$ a local minimum of $f$ on every line $PQ$, but with $f(Q) < f(P)$ for all $Q \neq P$, so that $P$ is not a local minimum of $f$.

This type of example can be made compatible with any non-negative Hessian with a zero direction (a line on which the Hessian quadratic form is $0$). If the Hessian is positive definite in the strict sense, with no zero directions, then of course the theorem is true and $P$ is a local minimum, but the argument for going from $P$ being a line-by-line relative minimum to being a full-neighborhood relative minimum is the same as the argument for $P$ being a relative minimum (not using any lines), and arguing "per line" does not help to solve the problem.

What is needed for a proof is not that specific to Hessians, or degree $2$, or finite dimensions. It is that the function can be written as $$ f(P+x) = f(P) + g(x)(1 + o(1)) $$ where $g(x)$ is a non-negative function in a neighborhood of $P$ and the $o(1)$ is as $|x| \to 0$.

Usually $g(x)$ is the lowest degree non-negative part of the Taylor expansion near $P$, therefore homogeneous (and of even degree), and then $g \geq 0$ in a neighborhood is the same as $g \geq 0$ everywhere. This is true if $g(x)$ is any non-negative Hessian. However, if there is a zero direction for $g$, then unless the difference $(f(P+x)-f(P))$ is equal to $g(x)$ (hence equal to $0$) in that direction the $1+o(1)$ vanishing relative error condition will be violated. The same is true if there is a curved zero locus for $g$ that goes through $P$, so to get a workable condition for a local minimum, $g$ should be $>0$ on a neighborhood of $P$.

Answer 2

Previous facts:

$D^kf(P)$ can be considered a $k$-linear function or a k-th form. In Taylor, is the second, namely:

$$ (D^2f(P))(h,h)=h^{\bf T}(D^2f(P))h=\sum_{i,j=1}^n(\partial_{ij}f(P))h_ih_j. $$

A definite positive quadratic form $q$ verifies that $\exists C>0\ \forall h\in{\Bbb R^n}:\ q(h)\ge C||h||^2$. Proof: $q$ has a maximum over the unit sphere, so $\exists C>0:\ q(h)\ge C$ when $||h||=1$. By homogeneity, the desired formula follows.

Now, suppose that $f$ is $C^3$. using Taylor:

$$ f(P+h)=f(P)+{1\over 2}(D^2f(P))(h,h)+{1\over 6}(D^3f(P_h))(h,h,h), $$

with $P_h$ in the segment between $P$ and $P+h$. The last term can be bound near $h=0$ by the continuity of $D^3f$:

$$ |{1\over 6}(D^3f(P_h))(h,h,h)|\le {1\over 6}\sum_{i,j,k=1}^n|\partial_{ijk}f(P_h)||h_ih_jh_k|\le M||h||^3, $$ and:

$$ f(P+h)-f(P)={1\over 2}(D^2f(P))(h,h)+{1\over 6}(D^3f(P_h))(h,h,h)\ge C||h||^2-M||h||^3>0 $$ for $h\ne 0$ near zero.