I have a function $F:\mathbb{R}^n\to[0,1]$ such that $$ F(\lambda) = \mathbb{E}_x[f(\lambda;x)] = \int f(\lambda;x)\mu(x)dx,$$ and I want to find $\tilde\lambda$ that maximizes F, i.e. $F(\tilde\lambda)=1$. Clearly, $\tilde\lambda$ is not unique. What is the best algorithm to find one of the $\tilde\lambda$?
For the moment I tried the stochastic gradient descent (with a random guess of $\tilde\lambda$ and with a discretization of $\mu(x)$) with moderate success when $n$ is small $\simeq 7$. I went for stochastic gradient descent for its ability to escape some local minima. What is the optimal algorithm when $n\simeq20$?
Notes:
- I can also evaluate the expectation value analytically, so my problem can be cast into a global optimization problem without any random part. Is there any advantage in exploiting the stochastic nature of my function? To the best of my knowledge, most global optimization algorithms have some stochastic part.
- I already know the maximum value (1), so in some sense my question is also a root-finding problem.
- I don't know the gradient of $F$, nor the gradient of $f$. However $F$ and $f$ are smooth.