Why is it sometimes possible to solve constrained optimization by directly taking the gradient and pretend as if the problem is unconstrained?

Question

Let $f(x) = x\log(x), x \in [0, \infty)$

I wish to find the minizer of this function.

Officially, one way to approach this problem is by using Lagrange duality. Set up a dual variable, solve a saddle point problem, check slater's condition, verify strong or weak duality, yadi yada.

But I cheat by directly taking the gradient of it and pretend that the problem is not constrained.

$f^\prime(x^\star) = \log(x^\star) + 1 = 0 \implies x^\star = 1/e$

Plug back in, yields

$f(x^\star) = (1/e)\log(1/e) = -1/e.$

An exact match: https://www.wolframalpha.com/input/?i=compute+minimum+of+f%28x%29+%3D+xlog%28x%29

So what happened? Why was I able to solve this constrained problem as if it was unconstrained and get the exact same answer? Is this cheating (meaning, is the Lagrange duality method the only correct way to solve constrained problems)?

Is there some general result that says when this can be done? For example, suppose the function is known to have an optimum point in the interior of its domain, then this is possible.

Answer 1

It's always true that the optimum of an inequality-constrained minimization problem is either

1) on the interior of the domain, so that the constrained minimizer is also an unconstrained minimizer; in this case the constraint is inactive;

2) on the boundary, where you're not at an unconstrained minimum, but all descent directions are "blocked" by the boundary; in this case the constraint is active.

This alternative is obvious geometrically, and generalizes to where you have multiple constraints, and some subset will be active at each optimum. The method of Lagrange multipliers lets you encode both situations in one clean equation, but it's a convenience; there's nothing fundamentally different about using the method of Lagrange multipliers vs. explicitly checking both cases. In fact many numerical algorithms for solving constrained optimization problems will alternate between updating a guess for which set of constraints are active, and then solving the problem without equality constraints, assuming the active set is correct.