Is the Lagrange multiplier $\lambda$ always positive?

Question

In the Lagrange function

$${\mathcal {L}}(x,y,\lambda )=f(x,y)-\lambda g(x,y),$$

is the Lagrange multiplier $\lambda$ term supposed to always be positive or can't it take negative values?

Answer 1

It need not be positive. In particular, when the constraints involve inequalities, a non-positivity condition may be even imposed on a Lagrange multiplier: KKT conditions.

Answer 2

The sign (and value) of $\lambda$ signifies what happens to the value of the extremum you find when you change the constraint by a tiny amount, say from $g(x,y)=0$ to $g(x,y)=\epsilon$. That's it. Positive or negative, large or small doesn't matter. If you have found a $\lambda$ (i.e. if you have found an $x$ and a $y$ for which a $\lambda$ exists), you have found an extremum.

Answer 3

Geometrically, the Lagrange multiplier method has the following interpretation:

1. We want a point on the level curve $g(x,y)=0$. This comes from differentiating with respect to $\lambda$ and setting to zero.
2. We want that, at the point $p$ under consideration, the level curve of $f(x,y)$ is tangent to the level curve $g(x,y)=0$. This is equivalent to the normal vectors to the level curves, given by the gradients, are parallel. In other words, we require the existence of a scalar $\lambda$ (necessarily different from zero if we assume a minimum of regularity for the level curves) such that $$\nabla_{p} f = \lambda\nabla_{p} g$$

From condition $2$, it is clear that there is no restriction telling us that $\lambda$ must e positive or negative. On the contrary, with this intuition it is easy to construct examples for both cases.

Answer 4

The idea behind the function $g$ is that we want a function such that $g(x,y)=0$ iff the constraint is satisfied. For instance, if our constraint is that we have to be on the unit circle, then a candidate for $g$ is $x^2+y^2-1$. But given any scalar $c$, if $g_1$ is a candidate, then $g_2=\lambda*g_1$ is also a candidate. For instance, $\frac{x^2+y^2-1}2$ would also define the unit circle. So there isn't a particular function $g$ that works, there is an entire family of functions that differ by scalar factors.

While any one of these functions satisfies the condition of characterizing the constraint, only one of them will have a gradient equal to the objective function's gradient. So what we do is simply pick any representative $g_1$, then take this $g_1$ as having been multiplied by the right parameter to yield the $g$ such that its gradient is equal to the objective function's gradient. Then, given that this parameter yields the correct $g$, we can calculate what the parameter must be.

Thus, the value of $\lambda$ depends on what $g$ we take as our original function. If we take $g(x,y)= \frac{x^2+y^2-1}2$, we'll get a different $\lambda$ than if we take $g(x,y)= x^2+y^2-1$. And if we take $g(x,y)= 1-x^2-y^2$, then the sign of $\lambda$ will be opposite to what we would get with $g(x,y)= x^2+y^2-1$.

So, no, $\lambda$ doesn't have to be positive. We are effectively setting its value by what we choose our $g_1$ to be, and we can make it anything we want (other than zero).

BTW, some people give the equation as ${\mathcal {L}}(x,y,\lambda )=f(x,y)+\lambda g(x,y)$, and of course the $\lambda$ as defined by that convention will have a sign opposite to the one defined by the convention you cite.