Reducing a Least sqare with equality and inequality constraints problem.

50 Views Asked by At

First, i know that there are a lot of topic about thoose kind of questions, but i thought about a reducing approach that i did not found in the litterature and i wander if the following steps are OK or if i'm making a horrible mistake.

Suppose you want to solve for $x \in \mathbb{R}^n$ the following problem :

$$ \min \frac{1}{2}x' A x - b'x$$

Under the constraints that $Ex = f$ and $Gx \ge h$, where $E$ and $G$ are matrices with $n$ columns and given number of rows $m_E,m_G$ such that $m_E < n$.

My thought are in 3 steps :

1. Treating the equality constraints first

Since $Ex = f$, we can say that $x$ belongs to an affine subspace which is parralel to the linear subspace given by the kernel of $E$. In my particular problem, this linear subspace has a dimension, say $d_{ker(E)}$, smaller than $n$.

Furthermore, I do known a particular solution, say $\lambda$, to $Ex=f$, which gives me the full specification of this affine space :

$$Ex = f \iff x \in S = \{x = \lambda + \sum\limits_{i=1}^{d_{Ker(E)}} y_i e_i, y \in \mathbb{R}^{d_{Ker(E}}\}$$

If we denote by $(e_i)$ a basis of $Ker(E)$.

2. Injecting it into the problem

Then, solving for $y$ under only the inquality constraint the folloiwng problem will be quivalent :

$$ \min \frac{1}{2}(\lambda + \sum\limits_{i=1}^{d_{Ker(E)}} y_i e_i)'A(\lambda + \sum\limits_{i=1}^{d_{Ker(E)}} y_i e_i) - b'(\lambda + \sum\limits_{i=1}^{d_{Ker(E)}} y_i e_i)$$

Which can be simplified by taking out all terms without $y$ to :

$$ \min \sum\limits_{i=1}^{d_{Ker(E)}} \frac{1}{2}(y_i e_i)'A(y_i e_i) + \lambda'A(y_i e_i) + (y_i e_i)'A\lambda - b'y_i e_i$$

3. Back to $x$

Doing this simple process, we reduced the dimension of the problem, and we supressed all equality constraints. The inquality constraints are of course adapted accordingly, and the basis of $Ker(E)$ can be use to get back $x$ from $y$. My question is then 2 parts :

Easy question : Is this simplification correct ?

Harder one : Is it usefull to pre-treat the problem like that before using a QP solver ?

PS : I'm not shure about the tags i used.. Feel free to change them.

1

There are 1 best solutions below

3
On BEST ANSWER

Looks correct, and sometimes you have to do this if the solver doesn't support equalities.

I would typically not recommend you to do it though if you have a solver which supports equalities. You possibly hide structure in the problem, convert a sparse problem to a dense problem, or introduce numerical issues. The solver knows best how to deal with the equalities.