Minimizing a quadratic function subject to a quadratic constraint

66 Views Asked by At

It is required to minimize the quadratic function

$f(\mathbf{x}) = \mathbf{x}^T Q \mathbf{x} + b^T \mathbf{x} + c $

Subject to the quadratic constraint

$ g(\mathbf{x}) = \mathbf{x}^T C \mathbf{x} + d^T \mathbf{x} + e = 0$

where $\mathbf{x} \in \mathbb{R}^n$, and it is assumed that matrix $C$ is positive definite.

My approach is based on the Lagrange multiplier method, so I define

$ L(\mathbf{x}) = f(\mathbf{x}) + \lambda \ g(\mathbf{x}) $

Then we want

$ \nabla L = \nabla f + \lambda \nabla g = \mathbf{0}$

which translates into

$ \nabla L = \mathbf{0} = 2 Q \mathbf{x} + b + \lambda \ (2 C \mathbf{x} + d ) $

I am not sure how to proceed from this point. One way I can think of is to isolate $\mathbf{x}$,

$\mathbf{x} = - ( 2 Q + 2 \lambda C )^{-1} (b + \lambda d ) $

Substituting this into $g(\mathbf{x}) = 0 $ gives

$ (b + \lambda \ d )^T ( 2Q + 2 \lambda C )^{-1} C ( 2 Q + 2 \lambda C)^{-1} (b + \lambda \ d ) - d^T (2 Q + 2 \lambda C )^{-1} (b + \lambda d ) + e = 0 $

But I have no idea how I would solve for $\lambda$ from this equation. Would it possible to apply the matrix inversion lemma (Woodbury matrix identity) to simplify this equation ?

Another way to go, which I think is better, is to eliminate $\lambda$

We have

$\mathbf{0} = 2 Q \mathbf{x} + b + \lambda \ (2 C \mathbf{x} + d ) $

So,

$ 2 Q \mathbf{x} + b = -\lambda \ (2 C \mathbf{x} + d ) $

If we define

$ E_j = e_1 e_j^T - e_j e_1^T, \ j = 2,3, \dots , n $

Then

$ (2 Q \mathbf{x} + b )^T E_j (2 C \mathbf{x} + d) = 0 , \ j = 2, 3, \dots, n$

This is a quadratic system of $(n-1)$ quadratic equation in the unknown vector $\mathbf{x} \in \mathbf{R}^n$, in addition to these, we have the $n$-th equation which is $g(\mathbf{x}) = 0 $.

Now we can use a quadratic system solver to obtain all possible solutions. (Mathematica should be able to solve this system). Having obtained all the possible solutions, we select the one that gives the minimum value of $f(\mathbf{x})$. I appreciate your comments, hints, and alternative solutions.