I've been wondering about the test to classify stationary points using the Hessian matrix. In the 2 variable case, things are fairly easy to deal with, I'm concerned with 3 or more variables. I've been told that when the Hessian is positive/negative definite, the stationary point is a minimum/maximum. Those make sense.
I've also been told that if all the eigenvalues are non-zero, and not all the same sign, then the point is a saddle point. Again, makes sense.
My problem is the next bit - supposedly, if any eigenvalue is zero, we must check higher order derivatives to be sure about the nature of the point. However, say we had a stationary point for a 3 variable function, for which one eigenvalue of the Hessian is 0, but the others differ in sign. Intuitively, I read that as meaning along one "principal direction", the curvature is flat, and along the others the curve is convex/concave. While I can't actually visualise this, it very much seems like a saddle point to me, given the curving in different directions. My question is whether that intuition holds any water, or if my understanding of the eigenvalues and eigenvectors of the Hessian isn't quite up to scratch?
To clarify, my knowledge of the Hessian is minimal (found out about it today), but I've read that the eigenvectors tell you the directions of principal curvature, and in the case of a 2 variable function, and its corresponding 3-dimensional surface, this just means the directions of maximal and minimal curvature. Because this case is 3 dimensional, there's no issue, since I can visualise all the situations, and there's no capacity for an ambiguous situation like the one I mentioned to arise.