Can a point $p$ be a maximum with undefined Hessian in $p$?

Question

If, for example $ U=\{(x,y): x^2+y^2\leq1\}$ and $f:U\rightarrow R$ has continuous second derivatives in $U$, is it possible for $p \in U$ to be a maximum when the Hessian of $f(x,y)$ in $p$ is indefinite?

Answer 1

Yes, it can. For instance, take $f(x,y) = -|y|-|x|$. The Hessian is not defined, nor is the gradient (however the sub-gradient exists), but $x=y=0$ is the maximum point.

Answer 2

Are you sure that you mean undefined and not indefinite?

If you mean undefined, I will agree to Ture.

If you mean indefinite, then the answer is no.

If the Hessian is negative definite or semi-negative definite you can have a maximum at that point.

If the Hessian is positive definite, semi-positive definite or indefinite, then you cannot have a maximum at that point.

But be careful if you have a semidefinite case. I.e. if you Hessian was semi-negative definite at a critical point, your critical point could be either a maximum or a saddle point. You cannot make out it just from the Hessian.

Edit:

It seems, that you like to compute the maxima and minima of $f$ on $U$? Then you have to be careful and do it in three steps.

First, you compute all critical points of $f$ in the interior of $U$. That means, you have to compute $\nabla f(p)=0$ for $p\in\mathring{U}=\{(x,y)\in\mathbb{R}^2~:~x^2+y^2<1\}$. Then you check for each critical point $p$ how the Hessian is definite.

Second, you have to compute the critical points of $f\mid_{\partial U}$. Since $\partial U=\{(x,y)\in\mathbb{R}^2~:~x^2+y^2=1\}$ is a compact manifold, you cannot do the same as on the interior. Define $g(x,y)=x^2+y^2$, then $1$ is a regular value of $g$ and $\partial U=g^{-1}(1)$. Then $p\in\partial U$ is a critical value of $f\mid_{\partial U}$ if there exists $\lambda\in\mathbb{R}$ such that $\nabla f(p)=\lambda \nabla g(p)$. Since $\partial U$ is compact, you get at least two critical points and you know, that the critical point of $f\mid_{\partial U}$ with the highest value is the maximum of $f\mid_{\partial U}$ and the critical point with the lowest value is the minimum of $f\mid_{\partial U}$. Sometimes it is useful to check the position of your critical points on the set $\partial U$. But very important is, that you cannot use the Hessian of $f$ to make out the kind of critical points of $f\mid_{\partial U}$!

Third, you have to compare your critical points on $\mathring{U}$ and on $\partial U$.