I have an intermediate knowledge of the calculus of variations: I can handle constraints in functional or integral forms and extrapolate to multiple variables and functions. If I dig in my notebooks I can even probably remember how to do variable endpoint problems and take the second variation. I have never, however, encountered anything regarding the handling of optimization of functionals in the following form:
$$ \int\mathcal{L}_1[\phi_j,\hat{A}_j\phi_j]dV\int\mathcal{L}_2[\phi_j,\hat{B}_j\phi_j]dV - \int\mathcal{L}_3[\phi_j,\hat{C}_j\phi_j]dV $$
Where $\phi_j$ are the functions to be optimized, and $\hat{A}_j,\hat{B}_j,\hat{C}_j$ are differential operators. Generality aside, my actual problem is a little simpler:
$$ \mathcal{S} = \int\bar{\psi}\hat{A}\psi dV\int\bar{\psi}\hat{B}\psi dV - \int\bar{\psi}\hat{C}\psi dV $$ $$ \frac{\delta\mathcal{S}}{\delta\psi} = 0 $$ With the constraints $$ \hat{K}\psi = 0\\ \int\bar{\psi}\psi dV = 1 $$ Where $\hat{K}$ is a (differential) operator separate from $\hat{A}, \hat{B}$ and $\hat{C}$, and these operators are first or second-order differential operators with no mixed derivatives. $\psi = \psi(x_1,x_2,x_3)$ lives in the space of square-integrable functions over all of $\mathbb{R}^3$. I should also note that this functional certainly has a minimum value of 0, which is given beforehand. To someone like me, this seems like a monster of a math problem...and that's kind of why I'm here. I'm wondering:
Is it possible to solve a problem like this? In particular, do there exist existence and uniqueness theorems for the solution(s)? Is this a variational calculus problem, or will I need even heavier machinery? If so, what is it? Can you point me to some literature that would help me characterize or solve problems like this in the future? I can't imagine that mathematicians have not worked on the optimization of arbitrary functionals, so I'm being optimistic.
Thank you so much for considering my question!