Solve KKT system for Quadratic Constrained Quadratic Program

1k Views Asked by At

I'm having trouble solving one of the possible cases that arise when solving the KKT conditions of the following problem:

We have the following optimization problem in $ \mathrm x \in \mathbb R^n$, with $\mathrm Q$ being an $n\times n$ positive definite matrix, $\mathrm p \in \mathbb R^n$,$\mathrm b \in \mathbb R^n$ and $\mathrm c \in \mathbb R$ .

$$\begin{array}{ll} \text{minimize} & \mathrm x^{\top} \mathrm Q \mathrm x + 2 \mathrm p^{\top} \mathrm x \\ \text{subject to} & \mathrm x^{\top} \mathrm Q \mathrm x + 2 \mathrm b^{\top}\mathrm x \leq \mathrm c \end{array}$$

Assigning multiplier $\mu \in \mathbb R$ to the constraint, I get the following KKT system:

$$ \nabla f(x) + \mu \nabla g(x) = 0 \iff 2 \mathrm Q \mathrm x + 2 \mathrm p + \mu \mathrm Q \mathrm x + 2 \mu \mathrm b = 0 $$ $$ \mathrm x^{\top} \mathrm Q \mathrm x + 2 \mathrm b^{\top} \mathrm x \leq \mathrm c $$ $$ \mu \geq 0 $$ $$ \mu(\mathrm x^{\top} \mathrm Q \mathrm x + 2 \mathrm b^{\top}\mathrm x - \mathrm c) = 0 $$

I have to consider 2 different cases: either $\mu = 0$ or $\mathrm x^{\top} \mathrm Q \mathrm x + 2 \mathrm b^{\top} \mathrm x - \mathrm c = 0$.

The first one is rather easy. If $\mu = 0$ then the first equation is reduced to

$$ \nabla f(x) = 0 \iff 2 \mathrm Q \mathrm x + 2 \mathrm p = 0 \iff \mathrm x = - \mathrm Q^{-1} \mathrm p $$

Plugging $\mathrm x $ in the second equation gives us the condition we have to check in order to validate this minimizer. $$ (- \mathrm Q^{-1} \mathrm p)^{\top} \mathrm Q(- \mathrm Q^{-1} \mathrm p) + 2 \mathrm b^{\top} (- \mathrm Q^{-1} \mathrm p) \leq \mathrm c $$

which, after manipulation and completing the square can be written as

$$ \frac{(\mathrm p - \mathrm b )^{\top}}{\sqrt{c + \mathrm b^{\top} \mathrm Q \mathrm b }} \mathrm Q^{-1} \frac{(\mathrm p - \mathrm b )}{\sqrt{c + \mathrm b^{\top} \mathrm Q \mathrm b }} \leq 1 $$

This means that when this condition is met, we can plug $x = - \mathrm Q^{-1} \mathrm p$ into the objective function to get the respective minimum. That is, $- \mathrm p^{\top}\mathrm Q^{-1} \mathrm p$.

My difficulty is the case when $\mathrm x^{\top} \mathrm Q \mathrm x + 2 \mathrm b^{\top} \mathrm x - \mathrm c = 0$. If I follow the same reasoning I get

$$ \mathrm x = \mathrm Q^{-1} (\frac{-2 \mathrm p - 2 \mu \mathrm b}{2 + \mu}) $$ But since $\mathrm x$ depends on $\mu$ I can't advance.