Constrained Optimization - Lagrange Multipliers (Example)

Question

Let $f(x, y, z) = xyz$

$h1(x, y, z) = x + y + z − 4$5

and $h2(x, y, z) = 2x − y$.

Goal:

Minimize $f(x, y, z)$ subject to $h1(x, y, z) = 0$ and $h2(x, y, z) = 0$.

First part: Show that every feasible point is regular.

Clear since $h1$ and $h2$ will always be linearly independent given that $z$ is zero.

Now:

1. Use the first order necessary conditions to find all candidates for local minimum points.

2. Compute the tangent spaces to all the candidates.

3. Use second order necessary and sufficient conditions to decide which of the points are indeed local minimum points.

I obtained the following system with $\lambda_1$ = lambda one and $\lambda_2$ = lambda two:

$yz - \lambda_1 - 2\lambda_2 = 0 $

$xz - \lambda_1 - \lambda_2 = 0$

$xy - \lambda_1 = 0$

$x+y+z-45 =0$

$2x-y =0$

Main questions:

Is this system correct? Will answering 2 involve simply determining $x,y,z$ in terms of $L1/L2$ and solving? Is there a general procedure for answering parts 2 and 3? If nothing else, how might one compute tangent spaces for candidates?

Thank you for taking the time to read this rather long question.

Answer 1

The Lagrangian of your problem is $L(x,y,z,\lambda_1, \lambda_2) = xyz - \lambda_1 (x+y+z-45) - \lambda_2 (2x-y)$ so your system of first-order conditions is almost correct: the second equation should be $xz - \lambda_1 + \lambda_2 = 0$.

I did not understand your justification that all feasible points are regular. Here you need to compare the gradients $\nabla h_1(x,y,z)$ and $\nabla h_2(x,y,z)$. In addition, they don't need to be linearly independent at all feasible points but only at solutions of the first-order conditions.

For each candidate $(x^*,y^*,z^*)$ the tangent space is defined as $$ \mathcal{K}(x^*,y^*,z^*) := \left\{ d \mid \nabla h_i(x^*,y^*,z^*)^T d = 0, \ i = 1, 2 \right\}. $$ It must be recomputed for each candidate. Finally, second-order necessary conditions require that the Hessian $\nabla^2 L(x^*,y^*,z^*,\lambda_1^*,\lambda_2^*)$ be positive semi-definite on $\mathcal{K}(x^*,y^*,z^*)$. This means that for all $d$ in $\mathcal{K}(x^*,y^*,z^*)$, $$ d^T \nabla^2 L(x^*,y^*,z^*,\lambda_1^*,\lambda_2^*) d \geq 0. $$ The second-order sufficient conditions require $> 0$ instead of $\geq 0$. Again, these conditions must be checked for each candidate. Note that the second derivatives of $L$ are only computed with respect to $x$, $y$ and $z$---the variables of the problem.

It's also possible that in some cases, the gradients $\nabla h_1$ and $\nabla h_2$ are not linearly independent but yet there exist solutions to the first-order conditions. In this case, there will be multiple possible choices of $\lambda^*$ and the second-order conditions take a slightly different form.