Connection between Euler-Lagrange equations and KKT

1k Views Asked by At

While studying calculus of variations I came across the following problem: $$ \min_{\substack{x\in X \\ y\in Y}} F(x,y) \text{ s.t. } L(x,y) =0 $$ where $F$ is a convex functional, $L$ is a differential operator and $X$, $Y$ are Banach spaces. When, for each $x$, there is a unique $y$ such that $L(x,y)=0$ we define the operator $\phi(x) = y$ so we can study the reduced problem: $$ \min_{x\in X} \tilde{F}(x) = \min_{x\in X} F(x,\phi(x)) $$

The necessary and sufficient condition for optimality is $$ \langle {\tilde {F}}^{\,\prime}(x) \,, x-\tilde{x}\rangle_{X^*\,X} \geq 0,\quad \forall \tilde x \in X. $$ The Euler-Lagrange equation is $\tilde F'(x) = 0$. I cannot convince myself that the Euler-Lagrange equation is the same as the KKTs for the full problem involving the adjoint: \begin{align*} \mathcal{L}_x(x,y,\lambda)&=0 \\ \mathcal{L}_y(x,y,\lambda)&=0 \\ \mathcal{L}_\lambda(x,y,\lambda)&=0 \end{align*}

where $\mathcal{L}(x,y,\lambda) = F(x,y) - \lambda L(x,y)$.

Can you please help me see the connection?

1

There are 1 best solutions below

0
On

Here is my hand wavy answer. I was wondering about the same thing and I think I arrived at the answer from chapters 7 and 9 in 1. This could all be easily formalised of course.

Suppose we are trying to minimize a function. The Euler-Lagrange(E-L) optimality conditions are derived by stating that perturbing a function at its optimum point in all feasible directions would not yield a smaller minimum. Therefore the Frechet differential in directions of its tangent space is zero. Said in another way, by moving along vectors in the tangent cone of the constraint set the function value would only increase or remain the same.

The KKT conditions is saying a similar story, but is like a dual version. It says that the negative gradient or direction of decrease of the function lies in the normal cone which is the dual of the tangent cone. Therefore the direction of decrease of the function has all its components perpendicular to the constraint set at the optimum point or lies in the normal cone. Or said in another way by moving along vectors in the tangent cone we would only increase the function or keep it the same just like in the E-L optimality conditions.

The E-L equation is stated in terms of directional derivatives. The KKT conditions are stated in terms of subdifferentials.

The figure shows the normal cone in green and shows in red (very badly drawn by me) the tangent cone directions. The Euler-Lagrange conditions are formulated in this tangent cone. The KKT in the normal cone

The figure shows the normal cone in green and shows in red (very badly added by me) the tangent cone directions. The Euler-Lagrange conditions are formulated in this tangent cone. The KKT in the normal cone.

Luenberger, D.G., Optimization by vector space methods (Series in Decision and Control), New York-London. Sydney-Toronto: John Wiley and Sons, Inc., XIII, 326 p. (1969). ZBL0176.12701.