I am trying to solve a variational problem which involves finding the function $g(x)$ subject to the BCs $g(0) = 0, g(1) =1$, is bounded and monotonic, and which minimises the functional:
$\mathcal{S}[f[g(x)]] = \int_0^1 f[g(x)] dx$
and the constraint, or the relationship between $f$ and $g$ is given by the ODE on $[0,1]$:
$f''(x) = (\lambda + g(x)) f(x)$ with BCs: $f(0) = f(1) = 0$
Since this ODE does not have a closed-form solution for general $g(x)$, I don't know how to express the variation in $f$ caused by a variation in $g$: could someone point me to references dealing with these types of constraints or share any ideas? I have tried googling things like "Lagrangian involving functional" or "constraint involving functional", etc. but to no avail.
I have tried solving your problem using the variational principle and the fundamental lemma of calculus of variations. As the development is quite long, I wrote it on a tablet and put it in a Google Drive folder:
Complete development on personal Google Drive
Here is a summary:
I call your functional $\mathcal{S[f[g(x)]]}$ $I[f, g]$. I define the functionnal $\tilde I [\epsilon, \eta ]$ such as, for all admissible variations $u, v$ and for all real $\epsilon, \eta$:
$$ \left.\frac{\partial I[f + \eta v, g + \epsilon u]}{\partial \epsilon}\right|_{\epsilon = 0} = 0 $$
I don't consider the partial derivative with respect to $\eta$ whch leads to a dead end.
Using integration by parts, I find:
$$\frac{\partial \tilde I [\epsilon, \eta]}{\partial \epsilon} = \left[u f' \right]_b^a - \int_0^1 u .u'.f''(g+\epsilon u) \ dx$$
Using the fundamental lemma and your link between $f$ and $g$, I deduce:
$$u'(g(x) + \lambda)f(x) = 0$$
So if $\lambda \in \mathbb{R}$ (note that you did not write the nature of $\lambda$ at the time I write this answer), the only acceptable solutions would be the trivial ones with $f(x) = 0$ because of your BCs.
Sorry for the late answer, I hope it helped. If I did a mistake I would be glad to know it and correct it.