Consider the optimization problem of maximizing $x$ subject to $x \leq 5$ and $\sqrt{x^2 + y^2} \geq 1$. if you examine the graph of the feasible region, you will see that all feasible directions for the feasible point $(-1, 0)$ yield a decrease in the objective. However, however small a region you examine around $(-1, 0)$, there is always a point above or below $(-1, 0)$ on the unit circle, which gives a better objective value. So is $(-1, 0)$ not a local maximum, despite no feasible direction causing an improvement in the objective function?
Has my thinking failed me somewhere? Or is it simply the case that sometimes, feasible directions can mislead you into thinking a point is optimal?
It will depend on the definition of search direction.
If you only consider straight lines to be search directions, then there is no line you can walk along that will increase the objective value locally because these lines are limited by tangent of circle which is vertical at this point. So there is no better solution reachable by a straight line without passing at least a piece of the forbidden circular region.
But if we are allowed to take non-linear routes we can reach better solutions in any circular neighborhood. For example a parabola $x=-1+t^2/4, y=t$
being very asymmetrically zoomed-in we can see this parabola search path squeezes between vertical line and the forbidden unit circle region :