**Intuition** behind Lagrange Multipliers

Question

Why does this method actually work?I learnt that a while ago and have been using it ever since. I apply it to almost every optimisation problem. This method just seemed too good to be true. Now, I wonder why this works. (The answer does not need to be rigorous, an intuitive sketch will work just fine for me.)

Answer 1

In the simplest case, there is a constraint curve $C:g(x,y)=0$ in the $xy$-plane. Above it is a surface $z=f(x,y)$. If all the points on $C$ are plugged into $f$, then you get some sort of path wondering around the hills and valleys of the surface. In a Lagrange problem, you want to find the highest (or lowest) elevation on that path.

If you look down at the $xy$-plane, you can see $C$ and also a bunch of concentric(-ish) level curves $k=f(x,y)$. Now you want find the level curve of $f$ with the largest $k$ that intersects $C$. So pick out a level curve and notice that it intersects $C$ a couple times. Now start increasing $k$. At some point, there will be a very last level curve that intersects $C$. In my mind the level curves are shrinking with increasing $k$ until that last curve is just touching $C$ and any larger $k$ gives a curve that misses $C$.

If these curves are smooth (the level curves and $C$) then the last time they touch, they are tangent to each other. Which means their slopes in the $xy$-plane are the same. Which means their normal vectors are parallel. Which means the gradients of the 2-variable parent functions are parallel. So we write $\nabla f = \lambda \nabla g$.

If this doesn't explain it well enough, try imagining me waving my hands about in an instructive fashion.