Lagrange multiplier understanding problem

Question

I do have a problem with the lagrange mutiplier method. I understand how it works for something like: maximize $f(x,y)$ subject to $g(x,y)=c$.

But how do I handle something like: Maximize f(x,y) subject to $g(x,y)=||(x,y)||^2 \leq 1$. What I did (and it worked for this particular exercise) is that i just set $g(x,y)=||(x,y)||-c$ for a $ c \in [0,1]$ and then it happened to be sufficient for just one c.

Does this always work? I saw a solution which first checked all points where the gradient of g(x,y) and f(x,y) is 0 and then conluded that the those are the only possible solutions on the inner points of the set which fullfils $||x|| \leq 1$

Can someone explain me how this is all related?

Thanks!

Answer 1

This is a two-dimensional analogue of something you are probably familiar with: Suppose a differentiable function $f:\mathbb R\rightarrow \mathbb R$ is restricted to an interval $[a,b]$ and you want to find the extrema there. Then, the procedure you follow is:

1. Find the local extrema in $(a,b)$ by checking the values of $x$ in $(a,b)$for which $f'(x)=0$ or $f'(x)$ is undefined.

2. Evaluating $f(a)$ and $f(b)$.

3. Selecting the largest and smallest value(s) from 1. and 2.

The procedure works because $f$ is differentiable, hence continuous on the compact set $[a,b]$.

Now, in your case, you have a (presumably differentiable) $f:\mathbb R^{2}\rightarrow \mathbb R$, restricted to the set of points $(x,y)$ for which $||(x,y)||^2 \leq 1$.

This set is just the unit disk and its interior, which is compact and so we may use the corresponding result for such functions on $\mathbb R^{2}$:

1. Find the local extrema in the interior of the region by locating the points $(x,y)$ for which $\nabla f(x,y)=0$ or $\nabla f(x,y)$ is undefined.

2. Evaluating the values of $f$ on the disk i.e. on the set of points $(x,y)$ for which $||(x,y)||^2 = 1$.

3. Selecting the largest and smallest value(s) from 1. and 2.

Part 2. can be tricky, but in this special case, it might be easier to consider $f(\cos \theta, \sin \theta)$ and the values of $\theta $ that make $f$ maximum/minimum.

Answer 2

The theory for inequality constraints, that is $g(x,y) \le 1$, is much more complicated than for equations like $g(x,y) = 1$. As start you could consult the Wikipedia on Karush–Kuhn–Tucker conditions.

However, in your simple case $f,g:\mathbb R^2\to\mathbb R$, it boils down to following: You can first try to find a maximizer in the interior, that is, $\nabla f(x,y)$ needs to be zero. If such interior optimizer does not exist, you can try to find one on the boundary. That is, the gradient of Lagrangian to the constraint $g(x,y)=1$ needs to be zero.

As for sufficient condition for a maximizer at the boundary: Let $(x^*,y^*)\in\mathbb R^2$ and $\lambda^* \ge 0$ such that $$ \nabla f(x^*, y^*) + \lambda^* \nabla g(x^*, y^*) = 0 $$ and $$ g(x^*,y^*)=1.$$ If the Hessian of the Lagrangian $$ \nabla^2 f(x^*, y^*) + \lambda^* \nabla^2 g(x^*, y^*) $$ is negativ definit. Then, $(x^*,y^*)$ is a maximizer.