I am facing some problem while revisting KKT multipliers. The inequalities $h(x)\leq 0$ are to be satisfied by $\mu h(x)$. Which is only possible if the gradients of $f$ and $g$ kisses and the gradient vectors have the same direction. Because in that case we are eliminating the inequality region by assuming if $x^{*}$ satisfies $h(x)$ then it has to be in the border line. Which effectively replace the inequality with an equality.
But What if the $x^{*}$ is inside the inequality region ? Not in the borderline ? Why there was no additive variable $\alpha$ introduced $h(x) +\alpha = 0$. I didn't ask this question when I read it first. Now I am not understanding this.
If $x*$ is inside the feasible region, $\mu=0$. You could add $\alpha$ to the constraint to make it an equality constraint, but then the problem no longer satisfies the Slater condition if $h$ is nonlinear, which means the KKT conditions are no longer sufficient.
For example, for $\min_x \{ x : x^2 \leq 1 \}$ the KKT conditions are $x+\lambda(2x-1)=0$, $\lambda(x^2-1)=0$, $x^2 \leq 1$, $\lambda \geq 0$, which is satisfied by $x=\lambda=0$.