If the Jacobian matrix is positive definite, does that imply that the optimization problem has a unique solution?

Question

My PhD adviser told me that if the Jacobian matrix of the optimality conditions is positive definite, then it implies that the optimization problem has a unique solution. I was wondering what is the basis for this statement, in other words, what theorem or proposition gives this statement.

Here is a simple example which illustrates the statement.

Let $a, b > 0$ be positive constants. Our goal is to compute the $(p, q)$ which maximizes the objective $$ ap - bp^2 + bpq - bq^2. $$ The first-order optimality conditions are: $$ a - 2bp + bq = 0, \\ bp - 2bq = 0. $$ If we take the Jacobian matrix of the vector-valued function which is the optimality conditions, it gives us the matrix $$ \begin{bmatrix} -2b & b \\ b & -2b \end{bmatrix}. $$

If I were to change the signs of the optimality conditions, I could get the Jacobian to be the positive definite matrix $$ \begin{bmatrix} 2b & -b \\ -b & 2b \end{bmatrix}. $$

What mathematical theorem or proposition tells us that because the Jacobian matrix is positive definite, therefore the optimization problem has a unique solution?

Answer 1

Positive definiteness of the Hessian is equivalent to convexity for twice differentiable functions, and convexity is what's really at work here. So, the theorem you want is

Theorem Let $f:U\to\mathbb{R}$ be strictly convex, where $U\subset \mathbb{R}^n$ is convex. Then if $f$ has a local minimum in $U$, it has a global minimum.

You'll observe that looks weaker than your advisor's claim might have sounded: there are convex functions with no minimum, the simplest example being $e^x$. But it's easy to find local minima of differentiable functions, certainly in your case when the first derivatives form a linear system, so then you can be sure of a global minimum.

Answer 2

So, let me explain you. If a function is defined on a convex set and derivative(on its domain) and also its second derivation is positive, then this function is convex.

An extremely easy example is $f(x)=x^2, x \in \mathbb{R}$

So for multivariate functions this theorem results in the fact that Jacobian is positive definite.

Roughly speaking, positive definite in matrices (in $\mathbb{R}^m$) is equivalent to being positive in $\mathbb{R}$.

Convex optimization by Steve Boyd, Click here is the best reference for your question.