To pre-context this question: I am studying the book Non-Linear Programming Theory and Algorithms Third Edition by MOKHTAR S. BAZARAA, et al. for an independent study course, and this book is a lot thicker than what I’m normally used to in theory, and due to time limitations we’re jumping everywhere in the book, so we’re skipping a lot of chapters here and there.
In the mentioned book, they define a Primal Non-Linear Problem as:
$$\min z = f(x)$$ $$\text{subject to:}\qquad\qquad\qquad\qquad\qquad\qquad$$ $$g(x)\le0$$ $$h(x)=0$$ $$x\in S$$
and the Lagrangian Duel Problem (Chapter Six) as:
$$\max w = \inf\{f(x)+u^Tg(x)+v^Th(x):x\in S\}$$ $$\text{subject to:}\qquad\qquad\qquad\qquad\qquad\qquad$$ $$u\ge0$$
In the same book, they define the Karush-Kuhn-Tucker (KKT) Conditions (Chapter Four) as the following:
For a Non-Linear problem, $$\min z = f(x)$$ $$\text{subject to:}\qquad\qquad\qquad\qquad\qquad\qquad$$ $$g_i(x)\le0\quad \text{ for } i=1,\ldots, m$$ $$h_j(x)=0\quad \text{ for } j=1,\ldots, l$$ $$x\in S$$
and a feasible solution $\bar x$, then the necessary conditions for problem optimality are:
$$1)\quad \nabla f(\bar x)+\sum_{i=1}^mu_i\nabla g_i(\bar x) + \sum_{j=1}^l v_j\nabla h_j(\bar x)=0$$ $$2)\quad u_ig_i(\bar x)=0 \quad \text{ for } i=1,\ldots, m$$ $$3)\quad u_i\ge0 \quad \text{ for } i=1,\ldots, m$$
One thing I had immediately noticed when playing around with some of the exercises was that $u_ig_i(\bar x)=0$ mimics the same behaviors of complementary slackness of a linear problem. From what I know about notation and theory from working on Network Programming (in that convention for that subject uses $u$ and $v$ for dual variables), would it be safe to say $u_i$ and $v_i$ would be the dual variables of the KKT conditions for the original problem? The book doesn’t really mention anything about I could find, but if so, then for either the Fritz-John or KKT optimality conditions we are not only checking if the found solution is primal feasible, but dual feasible when duality gap is achieved.
In addition, the objective function of the duel is extremely similar to the first KKT optimality condition. Of course, what the first optimality condition is seemingly checking is for values which all the gradient at each constraint and object function is zero, i.e. it’s at local/global optimal point. Is this assumption true, or am I off in my understanding in this relationship?
I eventually want to play around with this in my own time and build a non-linear algorithm to attempt to solve some non-linear models for fun, and I had been informed before that commercial non-linear solvers usually find a point that it thinks is optimal, and plugs it into the KKT conditions to check if it is, so if my idea about the relationships are correct, then it would further help my understanding for when I want to make my own solver.