Positive / negative (and so on) definite matrix. Confusion about the terms

Question

I have problems in understanding the terminology used in Sylvester's Criterion about the "sign" of a matrix. I got the "positive-definite", the "negative-definite", the "indefinite" and "non-definite" (the last one is when the determinant is zero, whilst the penultimate refers to two eigenvalues with different signs).

What it's really unclear are those two:

• positive semi-definite: "all its eigenvalues are non negative"
• positive semi-definite but non positive definite: "there is one zero eigenvalue and the rest are non negative".

I would really understand why one has to complicate the terminology in such a horrible way. Why shall we use "non negative"? Just say "Positive".

Also, what's the difference between "all its eigenvalues are non negative" and "there is one zero eigenvalue and the rest are non negative"? I don't get this.

If zero is counted as "neither negative nor positive" then wouldn't positive semi-definite mean positive definite?

It's all so messy.

Answer 1

When they say "all its eigenvalues are non negative" for a definite matrix for the term "positive-definite", they are trying to emphasize that there does not exist a zero within the matrix, and likewise with "positive semi-definite but non positive definite". These distinctions are important when one is using matrices, and their eigenvalues, to determine determine if a function is convex or not using the Hessian matrix, as determining convexity function of a Hessian matrix with eigenvalues that are equal to zero may leads to inconclusive results on where if the function is convex or not. In other words, unless there exists other strictly positive (or negative for concave function) eigenvalues, we couldn't make a conclusion about the function.

Though, you are correct in that it is messy. It would be better of them to simply say that it's another way to phrase the exact same thing, but there is an importance on defining it such terms if the author is leading up to using those for such cases

Answer 2

When talking about numbers, positive refers to things that are strictly greater than zero, while non-negative refers to things that are greater than or equal to zero. So the positive numbers are a subset of the non-negative numbers.

Similarly, positive definite matrices are ones whose eigenvalues are strictly positive, i.e. are all greater than zero, whereas positive semi-definite matrices are ones whose eigenvalues are non-negative, i.e. are greater than or equal to zero. As such, the set of positive definite matrices is a subset of the set of positive semi-definite matrices. Or in other words, all positive definite matrices are positive semi-definite (because everything greater than zero is also greater than or equal to zero), but not all positive semi-definite matrices are positive definite.

It's a small distinction, but it's very important in terms of the results that apply to each set.

For example, Sylvester's criterion states that a matrix $M$ is positive definite if and only if all of its leading principal minors are positive, i.e. if you take each $k \times k$ submatrix that starts at the top-left entry of $M$ then its determinant is positive. That means that you only need to calculate $n$ determinants to check whether $M$ is positive definite.

On the other hand, $M$ is positive semi-definite if and only if all of its principal minors are positive, i.e. the determinant of every $k \times k$ submatrix formed by taking a matching set of $k$ rows and columns from $M$. That means you need to check $2^n - 1$ determinants (every non-empty choice of values from $\{1, \ldots, n\}$) to find out whether $M$ is positive semi-definite.