Hessian of quadratic objective function

216 Views Asked by At

I have the quadratic function

$$f(\boldsymbol{x}) = \frac{3}{2} \left (x_{1}^{2}+x_{2}^{2} \right) + (1+a) x_{1} x_{2} - \left(x_{1} + x_{2} \right) + b$$

where $a, b \in \Bbb R$ are unknown parameters. Then,

$$f(\boldsymbol{x})=\frac{1}{2} \boldsymbol{x}^{\top}\left[\begin{array}{cc} 3 & 1+a \\ 1+a & 3 \end{array}\right] \boldsymbol{x}-\boldsymbol{x}^{\top}\left[\begin{array}{l} 1 \\ 1 \end{array}\right]+b$$

I am trying to check for what values of $a$ and $b$ a global minimizer will exist, and I saw the solution says the following:

The unique global minimizer exists if and only if the Hessian is positive definite, which holds if and only if $(1+a)^{2}<9$ (by Sylvester's criterion). Hence, the largest set of values of $a$ and $b$ such that the global minimizer of $f$ exists is given by $-4<a<2$ and $b \in \mathbb{R}$ (unrestricted).

Why is the Hessian of $f$ just the following?

\begin{bmatrix} 3 & 1+a \\ 1+a & 3 \end{bmatrix}

Which theorem does this come from? Or am I misunderstanding the solution?

1

There are 1 best solutions below

0
On BEST ANSWER

Given function is $$f(\boldsymbol{x})=\frac{3}{2}\left(x_{1}^{2}+x_{2}^{2}\right)+(1+a) x_{1} x_{2}-\left(x_{1}+x_{2}\right)+b$$

This yields $$ \nabla f(\mathbf{x}) = \left[ \begin{array}{c} 3 x_1 + (1 + a) x_2 - 1 \\ 3 x_2 + (1 + a) x_1 - 1 \end{array} \right] $$

The critical points of $f$ are at the points where $$ \nabla f(\mathbf{x}) = 0 $$

Then we find the Hessian matrix as $$ H = Hf(\mathbf{x}) = \left[ \begin{array}{cc} 3 & 1 + a \\ 1 + a & b \\ \end{array} \right] $$

A sufficient condition for global minimization of $f$ at a critical point $\mathbf{x}^\star$ is that $H$ is positive definite.

Sylvester's criterion: A necessary and sufficient condition for $H$ to be positive definite everywhere is that its principal minors should be positive.

This gives us: $$ \Delta_1 = 3 > 0 $$ $$ \Delta_2 = \mbox{det}(H) = 9 - (1 + a)^2 > 0 $$

$\mbox{det}(H) > 0$ also ensures that $f$ has a unique critical point $\mathbf{x^\star}$.

Thus, $f$ will have a global minimum at $\mathbf{x}^\star$ if $$ (1 + a)^2 < 9 \ \mbox{or} \ |1 + a| < 3 \ \mbox{or} -3 < 1 + a < 3 $$

Hence, if $a \in (-4, 2)$, then $f$ has a global minimum at $\mathbf{x}^\star$.