I have a nonlinear optimisation problem of the form:
$$ \min \:~ (a - \rho_{a})^2 + (b - \rho_{b})^2 + (c - \rho_{c})^2 $$
where $a,b,c$ are constants, and $\rho$ are different functions of around $30,000$ variables, say $x_{i}$. The nominator fo these functions includes a weighted sum of most of them, plus a weighted sum of their pairwise combinations, including squares. The denominator includes a type of "norm", this is the squared root of the squared sum of each of them. The key difference between the three $\rho$ functions is the subset of variables included. But there is huge overlap.
I also have lower and upper bounds for each of these variables.
Is there any hope I can achieve a meaningful optimisation? If so, would you recommend a particular language/optimisation tool? I'm open to anything really.
For that number of variables, the trick is to be careful and not choose an algorithm that requires the computation of a dense Hessian. There may be a better way to cache things, but a matrix-free inexact Newton method should work. Basically, it's Newton's method where the optimality system is solved inexactly using some kind of Krylov method like conjugate-gradient. Both trust-region and line-search variants exist, which should guarantee convergence. As far as the performance, it will depend greatly on how the eigenvalues of the Hessian are distributed. If they're well clustered, then it should work well. If they're not, it'll perform poorly. Figuring this out can be, well, a pain because for that many variables it's unlikely that you could compute the dense Hessian and just check. Really, the best option is to just code the derivatives and try the algorithms to see if they work.
As far as software, Optizelle implements the requisite algorithms and has interfaces to C++, Python, and MATLAB. It's also BSD licensed. Use the development version as it works much better. One of these days, I'll post some installers, which'll make installation easier. If you ask on the support forum, I'll do it sooner rather than later.
In any case, 30000 variables by itself isn't a problem. I've scaled certain problems out to half a billion variables, but they had a relatively compact spectra and I had a lot of computation to calculate the requisite derivatives.