Rule of Thumb: Using Lagrange multiplier or "one variable" function

Question

Let's simply say:

$f: \mathbb R^{2} \to \mathbb R, f(x,y):= xy$ and $h: \mathbb R^{2} \to \mathbb R, h(x,y):=x+y-1$

I am aware of the usual lagrange multiplier method, in other words:

Let $\lambda \in \mathbb R$, and $(x,y) \in \mathbb R^{2}$: $\nabla f(x,y) - \lambda \nabla h(x,y)=0$ etc.

However, I am repeatedly made aware of the fact that I could simply set $y=1-x$

and then define $\varphi: \mathbb R \to \mathbb R, \varphi(x):=x(1-x)$

it follows that $\varphi^{'}(x)=1-2x$ and setting this to $0$, we get $x=\frac{1}{2}$

and note: for $x<\frac{1}{2}: \varphi^{'}(x)>0$ and for $x>\frac{1}{2}: \varphi^{'}(x)<0$

Therefore $\varphi$ has a global maximum at $\frac{1}{2}$ and subsequently $f$ has a global maximum at $\frac{1}{2}$ under the constraint $h=0$.

General question: Is the lagrange multiplier method always equivalent to the "one variable" method I described above? If not, what are shortcomings of one method against another?

Answer 1

In the example you give us, we are identifying $y$ as a function of $x$. This is not always possible to do for all points satisfying the constraint simultaneously. For instance, if the constraint were $x^2 + y^2 = 1$, then the constraint function is $h(x,y) = x^2+y^2-1$, and it's not true that we can solve for $y$ globally as a function of $x$ because there has to be a unique value of $y$ for a given value of $x$.

In order to make this approach work, we would have to break up the curve of the constraint into the graphs of two functions: \begin{align*} y = +\sqrt{1- x^2} \qquad\text{and}\qquad y = -\sqrt{1-x^2}, \end{align*} and then also check separately the points $(\pm 1,0)$ since $|dy/dx|\to+\infty$ as $x\to\pm 1$. It's good to note that at each point where we can write $y$ as a function of $x$, we have ${\partial h\over\partial y}(x,y) \ne 0$. The Implicit Function Theorem (IFT) says that given a constraint equation of the form $h(x,y) = 0$, we can solve for $y$ as a function of $x$ locally, not necessarily globally if ${\partial h\over\partial y}(x,y) \ne 0$.

We might try to take our globally defined constraint and chop it up into regions where we can solve for one variable in terms of the other like we do above using the IFT, perform a bunch of one-dimensional optimizations, and then keep track of the points that "fall through the cracks" such as $(\pm 1,0)$ in the example we saw above.

Overall, as with many classes of problems in mathematics, it's helpful to acquaint yourself with plenty of examples and try different techniques to get a feel for which methods are simpler to implement for a given instance of the class of problem you're working on.

Answer 2

Yes, it is. But one should avoid Lagrange in case the restriction can be easily solved to one variable. In that case Lagrange often is an overkill.

BTW: As $x(1-x)$ is a parabola with zeroes $0$ and $1$ you don’t even need derivatives to determine the extremum.

Answer 3

Making the problem more general, suppose that you need to maximize or minimize $$f(x_1,x_2,\cdots,x_n)$$ subject to $m$ linear equality constaints $$g_i(x_1,x_2,\cdots,x_n)=0 \qquad (i=1,2,\cdots,m)\tag 1$$ and to $p$ nonlinear equality constaints $$h_i(x_1,x_2,\cdots,x_n)=0 \qquad (i=1,2,\cdots,p)\tag 2$$ using $(1)$, you can eliminate $m$ of the $x_i$'s and stay with the problem of maximizing or minimizing a function of $(n-m)$ variables with $p$ nonlinear constraints.

For sure, if $p=0$ as in your case, the problem becomes quite simple.