Quadratic function optimum not aligning with implementation.

Question

Let's define a quadratic function ($x$, $x_0$ and $g$ are vectors and $H$ is a symmetric matrix so, $H^T=H$): $$f(x) = \frac{(x-x_0)^TH(x-x_0)}{2}+g^T(x-x_0)$$

$$=>f(x)=\frac{x^THx}{2}+\frac{x_0^THx_0}{2}-\frac{x_0^THx}{2}-\frac{x^THx_0}{2}+g^Tx-g^Tx_0$$ Let's take the derivative of this quadratic function: $$f'(x)=Hx-Hx_0+g$$ If we set this to zero, we get: \begin{equation}x=x_0-H^{-1}g \end{equation}

Since quadratics are purely convex or purely concave, the local optima should be a global one as well. So, any $x_1$ that satisfies the last equation above should be either a global maxima or a global minima.

Now, let's take a practical example in python. In the code below, I take some values and demonstrate in a practical example that this value doesn't seem to be either the maxima or the minima of the quadratic. I've been scratching my head trying to figure this out all day. Can someone please help me find what I'm doing wrong?

import numpy as np
H=np.array([[  1.26237865e-01,  -7.00917553e+01],
            [ -7.00917553e+01,  -2.52888121e+05]])
g=np.array([ -5.27827861e+00,   3.00637734e+04])
x_0=np.array([ 27.2716913 ,   0.14728398])

# Since x_1 satisfies the last equation above, 
# it should be the maxima or the minima of the quadratic.
x_1 = x_0-np.linalg.solve(H,g)

def quad(x,y):
    delx = x-x_0[0]
    dely = y-x_0[1]
    return (H[0,0]*delx**2+H[1,1]*dely**2+2*H[0,1]*delx*dely)/2+g[0]*delx+g[1]*dely

# Both of the below expressions return True, meaning x_1 
# is not the maxima or the minima.

quad(x_1[0],x_1[1])>quad(x_0[0],x_0[1])
quad(x_1[0],x_1[1])<quad(130.71157427,0.24026742)

P.S. In the code, I also use the fact that: $$x^THx=H_{1,1}x_1^2+H_{2,2}x_2^2+2H_{1,2}x_1x_2$$ for a two dimensional symmetric $H$ matrix.

Answer 1

You have made some misstatements.

I will dispense with $x_0$, since it doesn't really change the thrust of the below, but serves as a distraction.

$\frac{1}{2}x^THx + g^Tx$ is convex if all eigenvalues of H are nonnegative, in which case any stationary point or local minimum is a global minimum.

It is concave if all eigenvalues of $H$ are nonpositive, in which case any stationary point or local maximum is a global maximum.

It is indefinite (neither convex nor concave) if $H$ has at least one negative eigenvalue and one positive eigenvalue. Stationary point of the unconstrained problem is a saddlepoint, and not a local or global minimum or maximum.

Your $H$ has eigenvalues -2.528881404269755e+05 and 0.145664840501185, and is therefore indefinite. Therefore, the stationary point you calculated is a saddlepoint. The global minimum objective value of this problem is $-\infty$, and the global maximum objective value is $\infty$, i.e., the objective function is unbounded below and above.

If you imposed constraints on the problem, there might be a finite local and.or global minimum or maximum, depending on the constraints.