Lagrange multipliers. Why should all terms evaluate to zero?

67 Views Asked by At

Imagine I am trying to optimise some function of the variables $x_i$ with Lagrange's method of undetermined multipliers and arrive at the following relation

$$ \sum_i c_i dx_i = 0 $$

I've always accepted the argument that each coefficient must evaluate to 0 as very intuitive, but I don't think I can prove it.

My idea is that if you want your problem to have a unique solution then the one with all $c_i = 0$ must be the one.

What is the exact reason behind the idea that all coefficients must be zero?

1

There are 1 best solutions below

0
On

The notation you are using is most likely to be confusing for you if you are not properly comfortable with the theory of differential forms, which I doubt you are if you are asking a question about Lagrange multipliers. Let us stick to traditional multivariable calculus notation.

We are looking at extremizing the scalar function $f$ subject to the scalar constraint $g=0$.

The first point is that the directional derivative of $f$ in the direction of a unit vector $u$ is $\nabla f \cdot u$. Therefore if $\nabla f \cdot u$ is positive for a $u$ in the tangent (hyper)plane of the surface $g=0$, then one can follow the direction of $u$ to increase $f$ and follow the direction of $-u$ to decrease $f$. Thus such a point cannot be an extremum.

The second point is that the tangent (hyper)plane of the surface $g=0$ is exactly the vectors which are perpendicular to $\nabla g$. Thus we need to have $\nabla f \cdot u = 0$ whenever $u$ is perpendicular to $\nabla g$. One can prove using linear algebra that this is equivalent to $\nabla f$ being parallel to $\nabla g$ (for instance one can use the rank-nullity theorem).

For a careful proof one should notice that we actually cannot typically move in the direction of the tangent (hyper)plane and stay on the surface $g=0$ (unless $g=0$ is a plane, or at least is locally a plane). But you can move in such a direction and then project back down to the surface and still find a point with a larger/smaller value of $f$.