Why use Bordered Hessian than "simple" Hessian as second derivative test?

Question

Why use Bordered Hessian than "simple" Hessian as second derivative test in a multi constrained optimization problem? The critical points are found from the Lagrangian so they follow the constraints. All you have to do is to check the convexity/concavity with Hessian in these critical points and tell whether it is local minimum or maximum. I find it much easier in calculations and to remember the rules of convexity/concavity, than these of bordered Hessian...

Answer 1

You don't "need" bordered Hessians. You can do as follows (which I suppose is equivalent to bordered Hessian, but also works with inequality constraints). But as discussed below, you can't just use the definiteness of the unmodified (full) Hessian of the objective function to ascertain the nature of a stationary point if any constraints are active (satisfied with equality) at that point.

Presuming the objective function and all constraint functions are twice continuously differentiable, then at any local minimum, the Hessian of the Lagrangian projected into the null space of the Jacobian of active constraints must be positive semidefinite. I.e., if $Z$ is a basis for the null space of the Jacobian of active constraints, then $Z^T*(\text{Hessian of Lagrangian})*Z$ must be positive semidefinite. This must be positive definite for a strict local minimum.

So the Hessian of the objective function in a constrained problem having active constraint(s) need not be positive semidefinite if there are active constraints.

All the preceding applies to local maximum (strict local maximum), if positive semidefinite is changed to negative semidefinite (negative definite).

Notes:

1) Active constraints consist of all equality constraints, plus inequality constraints which are satisfied with equality.

2) See the definition of the Lagrangian at https://www.encyclopediaofmath.org/index.php/Karush-Kuhn-Tucker_conditions .

3) If all constraints are linear, then the Hessian of the Lagrangian = Hessian of the objective function because the 2nd derivatives of linear functions are zero. But you still need to do the projection jazz if any of these constraints are active. Note that lower or upper bound constraints are particular cases of linear inequality constraints. If the only constraints which are active are bound constraints, the projection of the Hessian into the null space of the Jacobian of active constraints amounts to eliminating the rows and columns of the Hessian corresponding to those components on their bounds.

4) Because Lagrange multipliers of inactive constraints are zero, if there are no active constraints, the Hessian of the Lagrangian = the Hessian of the objective function, and the Identity matrix is a basis for the null space of the Jacobian of active constraints, which results in the simplification of the criterion being the familiar condition that the Hessian of the objective function be positive semidefinite at a local minimum (positive definite if a strict local minimum).