Optimization - What if solution of the lagrangian is the critical point of the constraint

Question

$x \in\mathbb R^n$

let $g(x)=a$ be a constraint on $f(x)$

We want to maximize f(x) so we form the lagrangian:

$L=f(x)-\lambda (g(x)-a)$

But what if the solution we obtain $x^*$ is a critical point of the $g(x)$

What does this imply? And why do we need to make sure the solution obtained is not a critical point of the $g(x)$. What is the point of "MAKING SURE" We end up with a maximizer of the function $f(x)$ at the end? Isn't our aim finding the $x$ that maximizes $f(x)$, why do we care whether it's a critical point of $g(x)$ ?

Answer 1

It's fine if a solution to the Lagrangian is a critical point of $g$. But it's also possible that $g$ has critical points that aren't solutions to the Lagrangian -- you need to check the values of $f$ at the $g$-critical point too.

It's like finding the maximum of a function $f: \mathbb R \rightarrow \mathbb R$ over an interval $[a,b]$. You find where $f'$ is zero but you also need to check the values at $a$ and $b$. It's possible you might have already found one or both of the endpoints as places where the derivative vanishes, but you always need to check $f$ there in addition. I'll bet you can sketch a function where the maximum occurs at an end point but $f'$ isn't zero there.

---

More formally, the method of Lagrange multipliers works because there's a theorem (often not stated explicitly) that says:

If $f$ is maximal at a point $p$, and $p$ is not a critical point of $g$, then there exists a solution to the Lagrange equation at $p$.

Let's call that theorem the "Lagrange Multiplier Theorem", so as not to confuse it with the Lagrange Theorem from group theory.

So "Necessary" in this context means "required to apply the Lagrange Multiplier theorem at that point", or in more detail "a condition that, if it's true at a point, and if it's also true that $f$ is maximal at that point, then there will be a solution to the Lagrange equation there".

Here's how I think about it: we have a whole set of points and we split them into two disjoint subsets, $A = \{x | g$ is regular at $x\}$ and $B = \{x | g$ is not regular at $x\}$. (We expect $A$ to be almost all of our set, and $B$ to be a few isolated points.). We know that there's some point, say $x_0$, where $f$ is maximal and we want to find it. If $x_0 \in A$, Largrange's theorem applies, so if we set-up the Lagrange equation and find all of its solutions, $x_0$ will be one of them. But it's also possible that $x_0 \in B$, where Lagrange doesn't apply, (i.e. we're not guaranteed that there is a solution to the Lagrange equation at $x_0$), so to deal with this possibility we just find all the points in $B$.

Answer 2

Let me give an example. Define $f(x)=x^3$, and define $g$ as $g(x)=0$ if $x\in[-1,1]$, $g(x) = (x+1)^2$ if $x<-1$, $g(x)=(x-1)^2$ if $x>1$. Note that $g$ is differentiable everywhere. Now consider the problem $\max \{ f(x) : g(x) = 0 \}$. The method of Lagrange only gives the point $x=0$ which is neither a minimum nor a maximum. Clearly $x=-1$ is a minimum and $x=1$ is a maximum, but these are missed by Lagrange's method since $\nabla g(x)=0$ at those points.