Suppose you have a multi-variable functional given as
$$ \int_{t_0}^{t_1} L(t,x,x', x'' ..., y, y', y'', ... ) \ dt $$
That you wish to to optimize [i.e find an $x$ and $y$ that maximize] but subject to a constraint
$$ \Omega(t,x,x', x'', ... y,y', y'' ... ) = 0 $$
What would be a general framework for approaching a problem like this?
Attempting to Generalize from Simpler Cases:
The finite-dimensional analog of this is to try to optimize a multivariable function
$$ F(x,y) $$
subject to a constraint $$G(x,y)=0$$. The standard approach is to use lagrange multipliers i.e. finding $x,y,\Lambda \in \mathbb{R} $ such that
$$ \nabla [ F(x,y)] = \Lambda \nabla [ G(x,y) ] $$ $$ G(x,y) = 0$$
In doing so, essentially the ingredients we need are
A notion of "functional gradient" which describes how $x,y$ ought to be perturbed to increase the value of our functional
A notion of a "functional tangent" which describes locally how a functional looks like
A notion of projection of the "functional gradient" to the "functional tangent"
Once you have these in place then a version of lagrange multipliers should be possible in the infinite dimensional case.
A dumb approach:
In theory lagrange multipliers are never "needed" they are merely elegant convenient and efficient.
Given The problem, minimize/maximize
$$ f(x_0, x_1, x_2 ... x_n) $$
subject to
$$ g(x_0, x_1, x_2 .. x_n) = 0 $$
We can observe there will always be a collection of functions $H_k$ such that if
$$ g(x_0, x_1, x_2 .. x_n) = 0 $$
then
$$ x_0 = H_k(x_1 .. x_n)$$
And therefore the problem of minimizing/maximizing $f(x_0, ... x_n)$ subject to the constraint can be restated as minimizing/maximizing $f(H(x_1 .. x_n) x_1 , ... x_n)$ that is completely unconstrained. It's obvious how to generalize this to our infinite dimensional case, and that works!
Unfortunately a more elegant "lagrangian" approach still eludes me.
More notes:
It's been some time since I posed this problem. I don't yet have a rigorous proof but I am fairly confident just introducing lagrange multiplier $\lambda_i$ for each constraint and functionally differentiating each one to create a system of couple differential equations is the correct approach here. It seems almost all multivariable techniques generalize in the obvious way to the infinite dimensional case so far (except the chain rule), so i would be surprised if this was any different.
I would start with the wiki page on Lagrange multipliers in Banach spaces and go from there. This theory has definitely been thoroughly explored, as it has lots of applications to Calculus of Variations.