At what point does the deviation from the minimum of a function become of first order rather than second order?

Question

I was reading Feynman's page on the Principle of Least Action, and he stated the following:

"That’s a possible way. But we can do it better than that. When we have a quantity which has a minimum—for instance, in an ordinary function like the temperature—one of the properties of the minimum is that if we go away from the minimum in the first order, the deviation of the function from its minimum value is only second order. At any place else on the curve, if we move a small distance the value of the function changes also in the first order. But at a minimum, a tiny motion away makes, in the first approximation, no difference (Fig. 19–8)" - the link is https://www.feynmanlectures.caltech.edu/II_19.html.

At what point would the deviation of the function from its minimum value become of first order rather than second order? Is this "transition" general to all functions with minima or does there exist a function where this change doesn't occur (i.e. deviation is first order and remains first order, or is second order and remains second order)?

Answer 1

A general second order expansion of a function $f(x)$ (which we assume to be analytic) about a point $x=a$ looks like $$ f(x) = f(a) + f'(a) (x-a) + \frac{1}{2} f''(a) (x-a)^2 + \cdots $$ So...

• Simplest (and, in my opinion, best answer): only at stationary points. $f'(a)=0$ only at a stationary point of $f(x)$. (That is, a local min, max, or an inflection point). Therefore, it is only really correct to ignore the linear order term completely if $a$ is a stationary point of $f$.

• Slightly more precise answer: "Near" stationary points. You may be able to approximate $f'(a)\approx 0$ if $x-a$ is small and $a$ is close to a stationary point. The reason this approximation may work is because if $a$ is sufficiently close to a stationary point, then $f'(a)$ will be close to its value at the stationary point, namely $0$. We also need $x-a$ to be small for the idea of a truncated Taylor expansion to work at all. This will be messy and you'll need to check on a case-by-case basis if this is really a good approximation. There are almost certainly easier approximations less prone to making errors, such as just keeping the linear term even if it is smaller than the quadratic term.

• Over-the-top answer: You can try to estimate how close to a stationary point you need to be with some assumptions. Just for fun, we can try to come up with a rough estimate for how close $a$ needs to be to the stationary point, assuming $f''(x)$ is approximately constant over the whole range of $x$ we are interested in, as follows. First, we expand around $a$ as above $$ f(x) = f(a) + f'(a) (x-a) + \frac{1}{2} f''(a)(x-a)^2 + \cdots $$ Then, we write $a=s + \delta$, where $s$ is the stationary point, and $\delta$ is a small deviation. Now note that we have two small quantities; $\delta$, and $x-a$. We want to use this expansion on the linear term $\sim O(x-a)$, so that we can take advantage of $f'(s)=0$. We also assume $f''(s)=f''(a)$ (in other words, the second derivative is approximately constant over the whole regime we're interested in). Working to second order in small quantities, we have \begin{eqnarray} f(x) &=& f(a) + \left[f'(s) + f''(a) \delta\right] (x-a) + \frac{1}{2} f''(a) (x-a)^2 + \cdots \\ &=& f(a) + \left[ \delta + \frac{1}{2} (x-a) \right] f''(a) (x-a) + \cdots \end{eqnarray} In order to be able to neglect the contribution of $f'(a)$ term, we evidently need to take $\delta \ll \frac{1}{2} (x-a)$. Or, in other words, for a given $\delta$, there is a limit on how small $|x-a|$ can be before you can no longer ignore the linear term. That is consistent with your normal intuition of Taylor series (lower order terms are relatively more important when $|x-a|$ is smaller); the special thing here is that you might be able to find a window where the quadratic term dominates the linear one ($|x-a|\gg \delta$) while remaining larger than the cubic and other higher order terms ($|x-a|\ll 1$). I want to emphasize this estimate is just meant to be illustrative. I did this quickly and for fun so it's possible I have lost track of some term that's important, and in any specific application you should check carefully that any approximation you make really does work and not rely on a general formula.

Answer 2

The absolute value function $f(x)=|x|$ is first-order near $x=0$, and so are many functions that involve it such as $\sin(|x|)$. These functions don't have a derivative at the minimum point.