I need C# code for solving constrainted non-linear least squares problems. I'm prepared to write the code myself, but I need to understand the algorithm first. Can anyone describe a constrained non-linear least squares algorithm for me, or point me to documentation elsewhere?
To elaborate, I know that the Levenberg-Marquardt algorithm is particularly well-documented, with many implementation examples available on the web. In its basic form, however, it does not support constraints. Nevertheless, some code libraries claim to offer implementations of the Levenberg-Marquardt algorithm which do support constraints (e.g. levmar). If I can find documentation for a technique that adds support for constraints to the Levenberg-Marquardt algorithm, I believe I would have what I need.
My specific requirement is that I need to support both equality and inequality constraints on linear functions of the variables. Simply enforcing bounds on the variables is insufficient.
Edit:
The question was faulted for "missing context or other details", so here is some further elaboration:
I need to create a procedure for tuning the theoretical models used for the operational planning of several industrial processes. The models will be tuned by adjusting model parameters to produce a good fit between the model predictions and historical observations. This is an optimization problem. Specifically, it is a constrained non-linear optimization problem with a sum-of-squares objective function that can only be evaluated numerically. I need a solver appropriate for that task.
The Levenberg-Marquardt algorithm looks like a good place to start. I have found descriptions of that algorithm on Wikipedia and in Numerical Recipes in C. However, those descriptions describe an unconstrained version. I don't know how to make the Levenberg-Marquardt algorithm (or any other non-linear least squares solver algorithm) respect constraints. The levmar code library (users.ics.forth.gr/~lourakis/levmar) claims to offer an implementation of the Levenberg-Marquardt algorithm that supports constraints, so it would seem possible to modify the algorithm to support constraints somehow. I can try to reverse-engineer the levmar code to figure it out, but I would prefer to approach this from a theoretical perspective first. Thus, I'm posting on the math forum to see if anyone can help me with the theory required to solve constrained non-linear least squares problems.
Note that I have already solved the problem using a generic non-linear solver (a solver that works with any objective function, not just sum-of-squares). I am not satisfied with the performance of that solver.