I've been studying constrained optimization problems and came across the geometric optimality condition, which states that if x^* is an optimal solution and there exists an arbitrary smooth curve $\gamma$ through S starting at x^* , i.e., $\gamma : [0, 1] \to S$ is a differentiable function with $\gamma(0) = x^*$, then:
$$ \frac{d}{dt} [f(\gamma(t))] \geq 0 $$
at (t = 0).
I've made some progress in understanding this condition. Specifically, I've realized that for a smooth curve \gamma(t) in the feasible set S:
$$ \frac{d}{dt} [f(\gamma(t))] = \nabla f(\gamma(t))^T \gamma'(t) $$
And when $\nabla f(\gamma(0)) = \nabla f(x^*) = 0$, the inequality holds as expected. However, I'm facing difficulty in understanding why $\nabla f(\gamma(t))^T \gamma'(0)$ should be greater than or equal to zero for $t = 0$.
My intuition is that for any possible vector $x - x^*$ or $\gamma'(0)$, it should hold that $\nabla f(\gamma(t))^T \gamma'(t) \geq 0\$ for (t = 0). Can someone help me clarify this aspect and possibly guide me through the proof of this condition? Additionally, any recommended references or reading materials on this topic would be highly appreciated. Thank you for your assistance!
Since $x^*$ is a local minimum, we have for all $t>0$ close enough to 0: $$ f\left(\gamma (0)\right)\le f\left(\gamma (0+t)\right) \\ \Leftrightarrow \\ 0\le f\left(\gamma (0+t)\right) -f\left(\gamma (0)\right) \\ \Leftrightarrow \\ 0\le\frac{f\left(\gamma (0+t)\right) -f\left(\gamma (0)\right)}{t} $$
And therefore in the limit $0\le \frac{d(f\circ \gamma)}{dt}(0)$.
As for reading materials - this heavily depends on your background knowledge and wanted level of generality. A compact script would be e.g. from the course Nonlinear Optimization from Mannheim, Chapter 11. As for an introductory book that covers most bases and many useful special cases, there'd be “Linear and Nonlinear Optimization” by Griva, Igor, Stephen G. Nash and Ariela Sofer.