I was taught the Lagrange multiplier method for one or more constraints but am struggling with the logic/intuition of an alternative that I know doesn’t work. Why isn’t it correct?
Suppose we want to find extrema of some function $f(x,y,z)$ given $g(x,y,z)=0$and $h(x,y,z)=0$.
In English, what we want the point(s) $(x,y,z)$ such that we achieve the maximum or minimum value of f while $(x,y,z)$ satisfy $g(x,y,z)=0$ AND $h(x,y,z)=0$.
So the correct way to solve is to find the solutions to:
$\nabla f=\lambda \nabla g + \mu \nabla h$
$g=0,h=0$
In other words, the gradient of f is some linear combination of the gradients of g and h.
As an example, say $(x_0,y_0,z_0)$ is a local max and gives the gradients for f, g and h as follows: $<3,1,1>=3<1,0,0>+1<0,1,1>$
If I remove a constraint $h=0$, shouldn’t $(x_0,y_0,z_0)$ still satisfy a local max given the one remaining constraint $g=0$? After all, I just removed a constraint but it was a local max and was on $g=0$ so it must still be a local max and satisfy the constraint. And yet we see that for $(x_0,y_0,z_0)$, $<3,1,1>=\lambda <1,0,0>$ yields no solution. So it is a local max given $g=0,h=0$, but not given the individual constraints. How come?
In other words, I’m asking why we can’t create simultaneous equations
$\nabla f= \lambda \nabla g$
$\nabla f= \mu \nabla h$
Shouldn’t the gradient of f have to parallel to the gradients of BOTH the constraints to be a local max for either of the constraints and therefore a local max for an ant traveling along the intersection of the constraints? Isn’t this a natural way to solve for extrema? How is a linear combination of the gradients sufficient?
Sorry for the wordy question, thanks in advance—I’ll take any intuitive explanation/counter-examples, etc. Thank you!!
This was addressed in another question but no explanation was given: Lagrange multipliers: More than one constraint
It seems to me that your confusion stems from your equivocating use of the term “local maximum”. In your example, since $\nabla f=(3,1,1)$ is not zero, the point $(x_0,y_0,z_0)$ is not a local maximum of $f$ in the unconstrained sense – it’s only a local maximum in the constrained sense that you can’t locally increase the value of $f$ while satisfying the constraints. But in the following, when you say “it was a local maximum so it must still be a local maximum”, you’ve switched the sense in which you’re using that term and are now assuming that the point is a local maximum in the unconstrained sense. If you use the term consistently, the paradox disappears – it’s not surprising that a point that’s a local maximum under a certain set of constraints is not a local maximum under another set of constraints. Specifically, in the example, if you drop the constraint $h=0$, $f$ can now be increased by moving in the direction $(0,1,1)$, which has positive scalar product with the gradients of $f$ and $h$ and is orthogonal to the gradient of $g$.