This questions is from the Kuhn-Tucker paper "Nonlinear Programming" in Section 2 Lemma 1. I don't understand how those conditions are necessary for a saddle point. I always thought that a saddle point was defined where the gradient is zero and the second derivative characterizes the shape of the saddle point.
I don't understand how the gradient could ever be non-zero. And assuming it could be non-zero why the gradient must be positive in one variable and negative in the other.
After this, everything else makes sense. Thanks in advance. A reference to something else that explains this would suffice, but I can't find one.
(This should be a comment, but got a bit long.)
The key is that the "saddle point" is not what you think it is. Note that right before Lemma 1 they defined their notion of a "Saddle Point problem" which in particular searches for a "saddle point" on a manifold with boundary given by the set $\{x \geq 0, u\geq0\}$. The same way that in a optimization problem that a maximizer or a minimizer may occur not in the interior, and instead on the boundary, a saddle point should also be allowed to occur on the boundary of this set. So they have a very specific generalized definition for a saddle point, which happens to coincide with the classical intuition in the interior of the set.
So in particular, the gradient in Lemma 1 can be nonzero if the solution of the saddle point problem happens on the boundary.
To get some intuition you can imagine the classical saddle $x^2 - y^2$, and looking at where the solution to the saddle point problem (as defined by the paper) lies over the sets of the form $\{x \geq x', y \geq y'\}$ where $x'$ and $y'$ are fixed (but arbitrary) numbers.