Algorithm for QP

58 Views Asked by At

Let $\mathbb{Q}^+ = \{ q \in \mathbb{Q} : q > 0 \}$. For $n \in \mathbb{N}^+$, let $a_1, \dots, a_n \in \mathbb{Q}$, $b_1, \dots, b_{n} \in \mathbb{Q}^+$ and $c_1, \ldots, c_n \in \mathbb{Z}$. Define $f : \mathbb{R}^n \to \mathbb{R}$ as $$f (x_1, \dots, x_n) = \sum_{i = 1}^n (a_i - x_i)^2$$

Suppose we want to minimize $f$ subject to the following constraints:

$$\sum_{i = 1}^n b_i x_i = d \text{, where $d\in \mathbb{Q}$ } \tag{LC}$$

$$x_i \geqslant c_i \text{ for all integers } 1 \leqslant i \leqslant n \tag{LB}$$


Algorithm

To solve this, I am currently using the following recursive algorithm:

Take Arguments: Objective, Linear Constraint, Lower Bounds

  1. minimize Objective subject to Linear Constraint only (using Lagrangian)
  2. if solution to (1.) satisfies Lower Bounds, return this solution
  3. otherwise, for each variable $x_i$ in Objective do the following:
    1. substitute $x_i = c_i$ into Linear Constraint and Lower Bounds

      call the results LC($i$) and LB($i$) respectively

    2. substitute $x_i = c_i$ into Objective, call the result O($i$)
    3. Apply Algorithm with

      Objective = O($i$), Linear Constraint = LC($i$) and Lower Bound = LB($i$)

  4. from solutions given by (3.), return the solution giving the smallest Objective

I run this algorithm with Objective = $f$, Linear Constraint = LC and Lower Bound = LB. Essentially, this minimizes without the LB constraint, and if that doesn't work, it “searches along the border” by substituting $x_i = c_i$ for each $1 \leqslant i \leqslant n$ and minimizing this instead (w.r.t. the remaining variables).


Questions

  1. Does this algorithm work (i.e. give the correct minimum)?

  2. Is there a faster way to find the minimum? (see below)


Context

I'm using Python (specifically SymPy) to find the exact minimum for this problem. Although there are much faster numerical algorithms, those aren't exact. I'm limited to methods available in Sympy (e.g., can't do symbolic minimization directly, but can solve equations) but am happy to implement any methods suggested here.