optimizing the weights in a system of linear equations with constraints

Question

For reference, I am solving a system of linear equations for a science project.

The system is of the form $G\ m=d$ where $d$ are data points, and $m$ are model elements.

I have an additional number of rows in the system, representing constraints (I suppose you could call them "soft constraints"), as for certain elements of the model (or linear combinations of those elements), I would like them to match or be close a certain value.

The additional rows are each weighted by a separate weighting factor $w_n$.

Now, I would like optimize for the values of those $w_n$, where there may be many constraints (say, $N=50$), each with a separate weight.

I cannot guarantee that there is some combination of weights where the misfit to data is at a global minimum.

However, What would be a good approach to search for combinations of weights that would minimize the misfit to data?, or perhaps, yield distributions of weights that produce better fits?

Thanks in advance.

Answer 1

If I understood correctly, you have two requirements for the model elements $m$: 1) it should fit the data $d$ and 2) it should lie close to the values you desire (which are imposed via additional constraints). Since these are competing requirements, you want to decide appropriate weightings.

The direct approach would be to plug it into an optimization solver: vary the weights $w_n$ to minimize the error between fitting the data and your additional constraints. You could use the least-squares solution formula to create the objective function, $$\min_w (G(G^TG)^{-1}(G^Td) - d)^2 $$

However, you may get weird results, or it might be difficult for the solver. I feel there is a subjectivity between how much you want to fit the data versus match your desirable values. I recommend starting with just fitting the data ($w_n = 0 \quad \forall n$), get the least-squares solution to that, see what values in the resulting $m$ vector are most unreasonably far from your desired values, and increasing weights on the relevant additional constraints. Obviously this is a slow, manual, iterative process, but it may work better when you don't have an objective way to evaluate the tradeoff between fitting the data and your desirable values.