Efficient SQP with more equality constraints than parameters

156 Views Asked by At

My question is both a math and (computer) programming one so answers related to either are fine.

Problem Setup

I have the nonlinear programming problem

$$ \begin{aligned} \min \;\; &f(X) \\ \text{s.t.} \;\; &g(X) \leq 0 \\ &h(X) = 0 \end{aligned} $$

where $X \in \mathbb{R}^k$, $g(X) \rightarrow \mathbb{R}^n$ and $h(X) \rightarrow \mathbb{R}^m$. Also $m > k$ so I have more constraints than parameters. I've already confirmed that SQP is well-suited for this problem and would like to continue using SQP, especially because it gives estimates for the KKT multipliers and they are physically meaningful in my problem. Since there are more equality constraints than parameters, most solvers will refuse to solve this problem (exit mode 2). MATLAB's fmincon with algorithm=sqp actually works quite well in this situation. It seems to be using a type of active-set method where it only activates a linearly independent set of constraints. I suspect this because the KKT/Lagrange multipliers output by fmincon are 0 for the dependent constraints meaning they're not active.

Computer Program Solution

I've been on a search for NLP solvers either directly in the Python ecosystem, or are written in another language but have interfaces and wrappers in Python. My work is open-source so closed-source and paid solutions aren't suitable. If anybody can bring an NLP solver that can handle this type of problem to my attention, that will work.

Mathematical Solution

Since I've pretty much convinced myself that this problem is primarily due to a gap in available NLP solvers in Python, I've begun to write my own. At a high level, here's the strategy:

  1. Create Lagrangian function for the problem to introduce the KKT multipliers.
  2. Linearize the problem to create a QP subproblem.
  3. Perform LU decomposition on the resulting set of linear equality constraints and remove any dependent constraints.
  4. Solve the QP subproblem using a QP solver or some other convex strategy.
  5. Reconstruct KKT/Lagrange multipliers that were removed from step 3.
  6. Repeat at step 1 using the current KKT/Lagrange estimate unless convergence.

This strategy actually solves my problem quite well, but it is EXTREMELY SLOW for medium-large problems >30 parameters. I'm using LU decomposition because I'm led to believe its a fairly efficient algorithm for Gaussian elimination. I can't help but feel there is a more efficient algorithm for handling cases where there are more constraints than parameters, especially since MATLAB's sqp seems to do it well. If anybody has recommendations on how I change my procedure at a high level, then that would be greatly appreciated.