Finding saddle point of indefinite quadratic function

231 Views Asked by At

There are lot of material on internet how to find the minimum or maximum solution for quadratic systems: \begin{equation} f(\mathbf{x}) = \mathbf{x}^T Q \mathbf{x} + \mathbf{y}^T(A\mathbf{x} - \mathbf{b}) + c, \end{equation}

where $\mathbf{y}$ is a regularization parameter. Most of them $Q$ is positive definite or negative definite which we can find the optimal solution using for example using Conjugate Gradient. I got a very specific problem, where \begin{align} Q = \begin{bmatrix} -I & 0\\ 0 & I \end{bmatrix} \end{align} is a indefinite matrix ($I$ is the identity).

How can I find the optimal solution for non-convex problems? (saddle point?)

1

There are 1 best solutions below

7
On BEST ANSWER

Calling $X = (x,y)$ and $Y = \mu = (\mu_1,\mu_2)$ we have

$$ L(X,\mu)-x'\cdot x+y'\cdot y +\mu_1'\cdot(A_{11}\cdot x+A_{12}\cdot y-b_1) + \mu_2'\cdot(A_{21}\cdot x+A_{22}\cdot y-b_2) $$

The stationary points for this lagrangian are given by the solutions for

$$ \cases{ -2 x'+\mu_1'\cdot A_{11} = 0\\ 2y'+\mu_2'\cdot A_{22} = 0\\ A_{11}\cdot x+A_{12}\cdot y-b_1 = 0\\ A_{21}\cdot x+A_{22}\cdot y-b_2 = 0 } $$

now solving for $\mu$

$$ \cases{ A_{11}A'_{11}\mu_1-A_{12}A'_{22}\mu_2 = 2b_1\\ A_{21}A'_{11}\mu_1-A_{22}A'_{22}\mu_2 = 2b_2 } $$

and substituting into

$$ x = \frac 12 A'_{11}\mu_1\\ y = -\frac 12 A'_{22}\mu_2 $$

we get the stationary points.