What does first and second approximations mean in this context?

Question

In the Feynman lectures on physics, Feynman in talking about the principle of least action, discusses how we should be able to find the true path $x(t)$ which has the least action, and the way to do it he says:

When we have a quantity which has a minimum—for instance, in an ordinary function like the temperature—one of the properties of the minimum is that if we go away from the minimum in the first order, the deviation of the function from its minimum value is only second order. At any place else on the curve, if we move a small distance the value of the function changes also in the first order. But at a minimum, a tiny motion away makes, in the first approximation, no difference (Fig. 19–8).

“That is what we are going to use to calculate the true path. If we have the true path, a curve which differs only a little bit from it will, in the first approximation, make no difference in the action. Any difference will be in the second approximation, if we really have a minimum.

The thing I understand from the diagram is that, near the minimum region the curve can be approximated as a parabola. Anywhere else on the curve, a tiny segment can be modeled as a straight line.

I do not understand what he means by first and second approximations, and that in the first approximation there is no difference, but there is a difference in the second approximation.

Answer 1

Any smooth function can be approximated by a polynomial in a small interval.

$$f(x)\approx a+bx+cx^2+dx^3+\cdots$$

For very small $x$, only the first term matters and the function can be seen as a constant

$$f(x)\approx a.$$ This is called the zero-th order approximation. It expresses the local value of the function.

For a little larger $x$, you must take into account the first term, and that yields a first order, linear approximation

$$f(x)\approx a+bx.$$

It expresses the local slope of the function.

A yet better approximation is obtained by adding more terms, yielding a second order, parabolic approximation.

$$f(x)\approx a+bx+cx^2.$$

It expresses the local concavity of the function.

The approximation coefficients can be estimated by taking the derivatives at $0$ [or another value that you like] and identifying:

$$f(x)=a+bx+cx^2+dx^3\cdots,f(0)=a$$ $$f'(x)=b+2cx+3dx^2\cdots,f'(0)=b$$ $$f''(x)=2c+3dx\cdots,f''(0)=2c.$$

Now in the case of an extremum, we have $f'(0)=b=0$, so that there is no first order approximation and one needs to jump to the second order.

$$f(x)\approx a\to f(x)\approx a+cx^2.$$

Here are constant (green), linear (red) and parabolic (black) approximations of the exponential and hyperbolic cosine functions.

Answer 2

What he means is that if we take some path $p$ and a variation $V$, and make another variation that we can choose to be small by parameterizing it as $\epsilon V$, then we can ask the questions (for each $n$):

What is $$ \lim_{\epsilon \to 0} \frac{A[p+\epsilon V] - A[p]}{\epsilon^n} ?$$

If this expression is non-zero for $n=1$ then we say that the deviation is first order. If it is zero for $n = 1$ but non-zero for $n=2$ then the deviation is second order. When Feynman says "only second order" he means "not first order"; that is, the deviation could be third or higher order as well as second order.

There is a real subtlety for the mathematical purist, concerning the choice of allowed "test" variations $\{V\}$.

At any rate "zero in first approximation" means that that limit is zero when $n=1$. If it looks like the familiar definition of a derivative, conceptually they are closely related, but here we mean to say that the limit is zero for all (suitable) test variations $V$.