Optimization: Are these equivalent forms of Taylor Series expansion for $f: \mathbb{R}^n \to \mathbb{R}$?

Question

I am currently using textbook on optimization.

On page 14, it reads:

Suppose that $f:\mathbb{R}^n \to \mathbb{R}$ is continuously differentiable and that $p \in \mathbb{R}^n$. Then we have that $$f(x+p) = f(x) + \nabla f(x+tp)^Tp$$ for some $t \in (0,1)$.

(Original text: Talyor's Theorem in Nocedal's Numerical Optimization)

This form of Taylor series expansion seems to be in conflict with the vast majority of Taylor series expansion I have seen, i.e.,

$$f(x) = f(\bar x) + \nabla f(\bar x)^T(x-\bar x) + \mathcal{o}(\|x-\bar x\|)$$

Are these the same Taylor series expansion? If so, how do we explain the $t \in (0,1)$ term in the first form, and the little-oh term in the second form?

Answer 1

The quoted text in yellow gives the zeroth-order order Taylor series for $f$ at $x$, with a remainder term. The version you wrote is the first-order Taylor series for $f$ at $\bar x$, with the remainder term written differently (and less explicitly).

The statement in yellow is a corollary of the mean value theorem. Let $g:[0,1] \to \mathbb R$ be the function defined by $$ g(s) = f(x + sp). $$ Clearly, $g$ is continuous on $[0,1]$ and differentiable on $(0,1)$. By the mean value theorem, there is a number $t \in (0,1)$ such that $$ \tag{$\clubsuit$} g'(t) = g(1) - g(0). $$ But $g(1) = f(x + p)$ and $g(0) = f(x)$, and by the chain rule $$ g'(t) = \nabla f(x + tp)^T p. $$ So, equation $(\clubsuit)$ implies that $$ f(x + p) = f(x) + \nabla f(x + tp)^T p. $$

Answer 2

$$f(x+p) = f(x) + \nabla f(x+tp)^Tp$$ is not the Taylor expansion. This is sort of like a Remainder result. Just like one dimensional case where we have the Mean Value Theorem $$ f(x+h)= f(x) + hf'(c)$$ for some $c$ between $x$ and $x+h.$