Strong and weak extrema

Question

I am confused about the "strength" of the two definitions. The definitions I use are the following:

Let $y$ be a function defined on the set $M$. Neighborhood (0. order) of the function $y$ is the set $U$ of functions $g$ on $M$ such that: $\exists\epsilon>0:\{g;|g(x)-y(x)|<\epsilon ; \forall x\in M\}\subset U$

Let $y$ be a function defined on the set $M$ where $f$ is also differentiable. Neighborhood (1. order) of the function $y$ is the set $U$ of functions $g$ defined and differentiable on $M$ such that: $\exists\epsilon>0: \{g; |g(x)-y(x)|<\epsilon;|g'(x)-y'(x)|<\epsilon; \forall x\in M\}\subset U$

The definition of the actual extremas are standard ($F(y)\geq F(y_0)$ for all functions within the neighbourhood)

Now an extremum is strong if it satisfies the 0. order neighborhod definition and weak if it satisfies the second definition.

My text then goes on saying Clearly every strong relative extremum is a weak relative extremum. The opposite does not hold

I don't see how that could be - it seems to me that it should be the exact opposite, since the 1. order has more restrictions. What am I missing?

Answer 1

I'm answering my year old question because a sudden realization struck me while dealing with $n-$terminal sets in probability theory. While David K's answer is correct, it did not help me understand, so I'll try to illustrate what helped my brain get this.

The confusing part is that, of course, if the function is in 1-order neighbourhood, it definitely is in 0 order neighbourhood. This is what we'd expect from a set with a stronger defining property. Yet, somehow, the claim is that $$ \text{strong extremum (0 order) }\Rightarrow \text{ weak extremum (1 order)}$$

Now that seems backwards, but it is true. They key is in (what David K did say in the comments) the "elimination of competitors". A function is the extremum if it's in some sense larger than its neighbourhood. 0-order neighbourhood has more functions, so the potential extremum has more competitors. Thus if it "wins" against more competitors, it definitely wins against their subset (1-order)

Answer 2

My intuitive understanding of this is that we define a neighborhood as a set of functions that satisfy some property $P.$ Now if we are fortunate, all the functions in this set also satisfy a certain property $Q,$ which allows us to identify a particular function as a relative extremum.

If we achieve this happy state, then if we place a stronger condition $P'$ on the functions in our neighborhood, that is, $P' \implies P,$ this may eliminate some functions from the neighborhood, but every function in the new neighborhood still satisfies $Q,$ which means our extremum is still an extremum. In other words, we don't make an extremal function any less extreme merely by eliminating some of the functions against which it is compared. It is only when we add functions to the neighborhood that we may introduce a function that violates $Q,$ thereby making our "extremum" no longer an extremum.

Answer 3

This is probably too late, but I figured I will take a crack ant answering this question and give a slightly different perspective.

The issue is that elimination process eliminates the extremum function in a real life problem. If the point $y^*$ maximizes/minimizes a functional $J$, let us say this value is some real $\alpha$. Now imposing a constraint on requiring the function to be differentiable now reduces your set of feasible solutions. The implication is not as you believe that it is winning against other competitors, not requiring differentiability may eliminate this current function hence changing the optimum. The point is that the maximum/minimum value attained cannot go down or up any further than that obtained from the $0$-order solution if there is one. If there exists an optimum under order-0 that is an greatest upper/least lower bounds attained by all feasible functions. That is the way I look at it.