Optimization with a constrained function

Question

Okay so I understand how to find points of extrema when for example,

We have $3x^2 + 2y^2 + 6z^2$ subject to the constaint $x+y+z=1$. I followed the method of the Lagrange multiplier and resulted in the point $(\frac13, \frac12, \frac16)$. I know this point is correct.

However, I don't know how to prove that it's a minimum value. Any ideas?

Answer 1

An elementary proof would be as follows: notice that by "completing the square",

$$3x^2 - 2x + 2y^2 - 2y + 6z^2 - 2z \\ = 3\left(\left(x - \frac{2}{6}\right)^2 - \left(\frac{2}{6}\right)^2\right) + 2\left(\left(y - \frac{1}{2}\right)^2 - \left(\frac{1}{2}\right)^2\right) + 6\left(\left(z - \frac{2}{12}\right)^2 - \left(\frac{2}{12}\right)^2\right)\\ = 3\left(x - \frac{2}{6}\right)^2 + 2\left(y - \frac{1}{2}\right)^2 + 6\left(z - \frac{2}{12}\right)^2 - \frac{1}{3} - \frac{1}{2} - \frac{1}{6}\\ \ge -1$$

(In case you're confused: $-\frac{1}{3} - \frac{1}{2} - \frac{1}{6} = -1$)

But $x + y + z = 1$, so we have

$$3x^2 + 2y^2 + 6z^2 -2 \ge -1$$ $$3x^2 + 2y^2 + 6z^2 \ge 1$$

It is quite evident (from the first inequality) that equality is achieved when $x = \frac{1}{3}, y = \frac{1}{2}, z = \frac{1}{6}$.

---

A more advanced method: using the Cauchy-Schwarz inequality, we have

$$(x + y + z)^2 \le \left(3x^2 + 2y^2 + 6z^2\right)\left(\frac{1}{3} + \frac{1}{2} + \frac{1}{6}\right)$$

But $x + y + z = 1$, so we can substitute this and simplify a bit to get:

$$3x^2 + 2y^2 + 6z^2 \ge 1$$

Equality holds when $3x = 2y = 6z$. From this, it can be trivially deduced that indeed, $(x,y,z) = \left(\frac{1}{3}, \frac{1}{2}, \frac{1}{6}\right)$ minimizes the expression with the given constrain.

While not exactly nor directly relevant to Lagrange Multipliers, this does provides a simpler alternative for certain optimization problems.

Answer 2

You can evaluate the entire Hessian matrix, and check the global (unconstrained) behaviour of the function. If the Hessian is positive definite (all eigenvalues >0) or negative definite (all<0), then an unconstraned minimum is also a constrained minimum (the same for maximum). This is the case with your question, because the hessian is just $2\operatorname{diag}(3,2,6) $.

If it's eigenvalues have mixed signs, then you have more work to do... you need to project the Hessian to the tangent plane to the constrained. You do that with the projection operator $P_{ij}=\delta_{ij}-n_in_j$ where $n$ is the normalized normal (which you obtain by taking the gradient of your constraint).

Answer 3

The bordered Hessian matrix for a function $f(x,y,z)$ and constraint $g(x,y,z)=0$ is: $$\mathbf{H}(x,y,z,\lambda) = \begin{bmatrix} 0 & g_x & g_y & g_z\\ g_x & f_{xx}+\lambda g_{xx} & f_{xy}+\lambda g_{xy} & f_{xz}+\lambda g_{xz} \\ g_y & f_{xy}+\lambda g_{xy} & f_{yy}+\lambda g_{yy} & f_{yz}+\lambda g_{yz} \\ g_z & f_{xy}+\lambda g_{xz} & f_{yz}+\lambda g_{yz} & f_{zz}+\lambda g_{zz} \end{bmatrix}$$ Which is the Hessian for the lagrangian. For your case $f(x,y,z)=3x^2+2y^2+6z^2$ and $g(x,y,z)=x+y+z-1$: $$\mathbf{H}(x,y,z,\lambda) = \begin{bmatrix} 0 & 1 & 1 & 1\\ 1 & 6 & 0 & 0 \\ 1 & 0 & 4 & 0 \\ 1 & 0 & 0 & 12 \end{bmatrix}$$ Which evaluated at your critical point is equal (being constant) and being the determinant equal to $-144$ is a minimum.

Answer 4

We can also make a geometric interpretation. The "level surfaces" of the function, $ \ 3x^2 + 2y^2 + 6z^2 \ = c \ , $ are triaxial ellipsoids centered on the origin. The tilted plane $ \ x + y + z \ = \ 1 \ $ passes through the first octant, where there is a "smallest" ellipsoid which is just tangent to this plane. (This is what the Lagrange-multiplier method determines.) Any larger ellipsoid, corresponding to larger values of the function, intersects the plane in a curve, giving an infinite number of possible values of the coordinate variables satisfying the constraint equation; there is no upper limit to the "size" of such ellipsoids (represented by the value of $ \ c \ $ ). Thus, the minimum value of the function, subject to the constraint, has been determined.

I have made the ellipsoid a trifle too large here to make it evident where the tangent point is. Larger ellipsoids would emerge further "through the constraint plane".